Skip to content

Commit

Permalink
fix #71 and #73: add support for outdir template and enhance docs
Browse files Browse the repository at this point in the history
  • Loading branch information
Thomas von Dein authored and TLINDEN committed Feb 10, 2024
1 parent 612ed2a commit 5a2c277
Show file tree
Hide file tree
Showing 6 changed files with 279 additions and 4 deletions.
4 changes: 3 additions & 1 deletion config.go
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,8 @@ const (

DefaultAdNameTemplate string = "{{.Slug}}"

DefaultOutdirTemplate string = "."

// for image download throttling
MinThrottle int = 2
MaxThrottle int = 20
Expand Down Expand Up @@ -128,7 +130,7 @@ func InitConfig(output io.Writer) (*Config, error) {
// Load default values using the confmap provider.
if err := kloader.Load(confmap.Provider(map[string]interface{}{
"template": template,
"outdir": ".",
"outdir": DefaultOutdirTemplate,
"loglevel": "notice",
"userid": 0,
"adnametemplate": DefaultAdNameTemplate,
Expand Down
90 changes: 89 additions & 1 deletion kleingebaeck.1
Original file line number Diff line number Diff line change
Expand Up @@ -195,7 +195,7 @@ Be careful if you want to change the template. The variable is a
multiline string surrounded by three double quotes. You can left out
certain fields and use any formatting you like. Refer to
<https://pkg.go.dev/text/template> for details how to write a
template.
template. Also read the \s-1TEMPLATES\s0 section below.
.PP
If you're on windows and want to customize the output directory, put
it into single quotes to avoid the backslashes interpreted as escape
Expand All @@ -204,6 +204,94 @@ chars like this:
.Vb 1
\& outdir = \*(AqC:\eData\eAds\*(Aq
.Ve
.SH "TEMPLATES"
.IX Header "TEMPLATES"
Various parts of the configuration can be modified using templates:
the output directory, the ad directory and the ad listing itself.
.SS "\s-1OUTPUT DIR TEMPLATE\s0"
.IX Subsection "OUTPUT DIR TEMPLATE"
The config varialbe \f(CW\*(C`outdir\*(C'\fR or the command line parameter \f(CW\*(C`\-o\*(C'\fR take a
template which may contain:
.ie n .IP """{{.Year}}""" 4
.el .IP "\f(CW{{.Year}}\fR" 4
.IX Item "{{.Year}}"
.PD 0
.ie n .IP """{{.Month}}""" 4
.el .IP "\f(CW{{.Month}}\fR" 4
.IX Item "{{.Month}}"
.ie n .IP """{{.Day}}""" 4
.el .IP "\f(CW{{.Day}}\fR" 4
.IX Item "{{.Day}}"
.PD
.PP
That way you can create a new output directory for every backup
run. For example:
.PP
.Vb 1
\& outdir = "/home/backups/ads\-{{.Year}}\-{{.Month}}\-{{.Day}}"
.Ve
.PP
Or using the command line flag:
.PP
.Vb 1
\& \-o "/home/backups/ads\-{{.Year}}\-{{.Month}}\-{{.Day}}"
.Ve
.PP
The default value is \f(CW\*(C`.\*(C'\fR \- the current directory.
.SS "\s-1AD DIRECTORY TEMPLATE\s0"
.IX Subsection "AD DIRECTORY TEMPLATE"
The ad directory name can be modified using the following ad values:
.IP "{{.Price}}" 4
.IX Item "{{.Price}}"
.PD 0
.IP "{{.ID}}" 4
.IX Item "{{.ID}}"
.IP "{{.Category}}" 4
.IX Item "{{.Category}}"
.IP "{{.Condition}}" 4
.IX Item "{{.Condition}}"
.IP "{{.Created}}" 4
.IX Item "{{.Created}}"
.IP "{{.Slug}}" 4
.IX Item "{{.Slug}}"
.IP "{{.Text}}" 4
.IX Item "{{.Text}}"
.PD
.PP
It can only be configured in the config file. By default only
\&\f(CW\*(C`{{.Slug}}\*(C'\fR is being used, this is the title of the ad in url format.
.SS "\s-1AD TEMPLATE\s0"
.IX Subsection "AD TEMPLATE"
The ad listing itself can be modified as well, using the same
variables as the ad name template above.
.PP
This is the default template:
.PP
.Vb 7
\& Title: {{.Title}}
\& Price: {{.Price}}
\& Id: {{.ID}}
\& Category: {{.Category}}
\& Condition: {{.Condition}}
\& Created: {{.Created}}
\& Expire: {{.Expire}}
\&
\& {{.Text}}
.Ve
.PP
The config parameter to modify is \f(CW\*(C`template\*(C'\fR. See example.conf in the
source repository. Please take care, since this is a multiline
string. This is how it shall look if you modify it:
.PP
.Vb 2
\& template="""
\& Title: {{.Title}}
\&
\& {{.Text}}
\& """
.Ve
.PP
That is, the content between the two \f(CW"""\fR chars is the template.
.SH "SETUP"
.IX Header "SETUP"
To setup the tool, you need to lookup your userid on
Expand Down
66 changes: 66 additions & 0 deletions kleingebaeck.go
Original file line number Diff line number Diff line change
Expand Up @@ -55,13 +55,79 @@ CONFIGURATION
multiline string surrounded by three double quotes. You can left out
certain fields and use any formatting you like. Refer to
<https://pkg.go.dev/text/template> for details how to write a template.
Also read the TEMPLATES section below.
If you're on windows and want to customize the output directory, put it
into single quotes to avoid the backslashes interpreted as escape chars
like this:
outdir = 'C:\Data\Ads'
TEMPLATES
Various parts of the configuration can be modified using templates: the
output directory, the ad directory and the ad listing itself.
OUTPUT DIR TEMPLATE
The config varialbe "outdir" or the command line parameter "-o" take a
template which may contain:
"{{.Year}}"
"{{.Month}}"
"{{.Day}}"
That way you can create a new output directory for every backup run. For
example:
outdir = "/home/backups/ads-{{.Year}}-{{.Month}}-{{.Day}}"
Or using the command line flag:
-o "/home/backups/ads-{{.Year}}-{{.Month}}-{{.Day}}"
The default value is "." - the current directory.
AD DIRECTORY TEMPLATE
The ad directory name can be modified using the following ad values:
{{.Price}}
{{.ID}}
{{.Category}}
{{.Condition}}
{{.Created}}
{{.Slug}}
{{.Text}}
It can only be configured in the config file. By default only
"{{.Slug}}" is being used, this is the title of the ad in url format.
AD TEMPLATE
The ad listing itself can be modified as well, using the same variables
as the ad name template above.
This is the default template:
Title: {{.Title}}
Price: {{.Price}}
Id: {{.ID}}
Category: {{.Category}}
Condition: {{.Condition}}
Created: {{.Created}}
Expire: {{.Expire}}
{{.Text}}
The config parameter to modify is "template". See example.conf in the
source repository. Please take care, since this is a multiline string.
This is how it shall look if you modify it:
template="""
Title: {{.Title}}
{{.Text}}
"""
That is, the content between the two """ chars is the template.
SETUP
To setup the tool, you need to lookup your userid on kleinanzeigen.de.
Go to your ad overview page while NOT being logged in:
Expand Down
87 changes: 86 additions & 1 deletion kleingebaeck.pod
Original file line number Diff line number Diff line change
Expand Up @@ -55,14 +55,99 @@ Be careful if you want to change the template. The variable is a
multiline string surrounded by three double quotes. You can left out
certain fields and use any formatting you like. Refer to
L<https://pkg.go.dev/text/template> for details how to write a
template.
template. Also read the TEMPLATES section below.

If you're on windows and want to customize the output directory, put
it into single quotes to avoid the backslashes interpreted as escape
chars like this:

outdir = 'C:\Data\Ads'

=head1 TEMPLATES

Various parts of the configuration can be modified using templates:
the output directory, the ad directory and the ad listing itself.

=head2 OUTPUT DIR TEMPLATE

The config varialbe C<outdir> or the command line parameter C<-o> take a
template which may contain:

=over

=item C<{{.Year}}>

=item C<{{.Month}}>

=item C<{{.Day}}>

=back

That way you can create a new output directory for every backup
run. For example:

outdir = "/home/backups/ads-{{.Year}}-{{.Month}}-{{.Day}}"

Or using the command line flag:

-o "/home/backups/ads-{{.Year}}-{{.Month}}-{{.Day}}"

The default value is C<.> - the current directory.

=head2 AD DIRECTORY TEMPLATE

The ad directory name can be modified using the following ad values:

=over

=item {{.Price}}

=item {{.ID}}

=item {{.Category}}

=item {{.Condition}}

=item {{.Created}}

=item {{.Slug}}

=item {{.Text}}

=back

It can only be configured in the config file. By default only
C<{{.Slug}}> is being used, this is the title of the ad in url format.

=head2 AD TEMPLATE

The ad listing itself can be modified as well, using the same
variables as the ad name template above.

This is the default template:

Title: {{.Title}}
Price: {{.Price}}
Id: {{.ID}}
Category: {{.Category}}
Condition: {{.Condition}}
Created: {{.Created}}
Expire: {{.Expire}}

{{.Text}}

The config parameter to modify is C<template>. See example.conf in the
source repository. Please take care, since this is a multiline
string. This is how it shall look if you modify it:

template="""
Title: {{.Title}}

{{.Text}}
"""

That is, the content between the two C<"""> chars is the template.

=head1 SETUP

To setup the tool, you need to lookup your userid on
Expand Down
8 changes: 7 additions & 1 deletion main.go
Original file line number Diff line number Diff line change
Expand Up @@ -112,11 +112,17 @@ func Main(output io.Writer) int {
slog.Debug("config", "conf", conf)

// prepare output dir
err = Mkdir(conf.Outdir)
outdir, err := OutDirName(conf)
if err != nil {
return Die(err)
}

err = Mkdir(outdir)
if err != nil {
return Die(err)
}
conf.Outdir = outdir

// used for all HTTP requests
fetch, err := NewFetcher(conf)
if err != nil {
Expand Down
28 changes: 28 additions & 0 deletions store.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,36 @@ import (
"runtime"
"strings"
tpl "text/template"
"time"
)

type OutdirData struct {
Year, Day, Month string
}

func OutDirName(conf *Config) (string, error) {
tmpl, err := tpl.New("outdir").Parse(conf.Outdir)
if err != nil {
return "", fmt.Errorf("failed to parse outdir template: %w", err)
}

buf := bytes.Buffer{}

now := time.Now()
data := OutdirData{
Year: now.Format("2006"),
Month: now.Format("02"),
Day: now.Format("01"),
}

err = tmpl.Execute(&buf, data)
if err != nil {
return "", fmt.Errorf("failed to execute outdir template: %w", err)
}

return buf.String(), nil
}

func AdDirName(conf *Config, advertisement *Ad) (string, error) {
tmpl, err := tpl.New("adname").Parse(conf.Adnametemplate)
if err != nil {
Expand Down

0 comments on commit 5a2c277

Please sign in to comment.