Commit Graph

81 Commits

Author SHA1 Message Date
Dimitri Herzog 720e6550be chore(test): fix flaky test 2023-09-14 15:03:37 +02:00
Dimitri Herzog 72d747c16f
feat: API-first approach for REST interface (#1129)
* feat: embed OpenAPI definition file

* feat: use OpenAPI generated server and client

* feat: provide OpenAPI interface documentation

* chore(test): add additional tests
2023-09-09 19:30:55 +02:00
Dimitri Herzog 245bb613df
fix: don't convert regex from punycode to ASCII (#1126) (#1131)
* fix: don't convert regex from punycode to ASCII (#1126)

* chore(test): add additional tests
2023-09-08 18:11:26 +02:00
DerRockWolf c112e86740
feat: add upstream strategy `strict` (#1093) 2023-08-21 09:50:23 +02:00
ThinkChaos cfc3699ab5 feat: support multiple hosts files 2023-07-07 09:16:20 -04:00
ThinkChaos ea95d36f37 fix: make domain validation in list parser more lenient 2023-04-14 16:20:54 -04:00
ThinkChaos 8253075f28
test: remove focus in list_cache suite (#990)
Accidentally left `FIt` when committing the test.
2023-04-13 09:10:40 +02:00
mandrav 015b565137
Add maxErrorsPerFile blocking configuration (#986)
* Add maxErrorsPerFile blocking configuration

The default max errors per file of 5 is too small IMHO.
This commit makes this number user-configurable.

* squash: fix lint

* squash: docs

* squash: change type to int to allow -1

* squash: test that the `maxErrorsPerFile` is actually used

---------

Co-authored-by: ThinkChaos <ThinkChaos@users.noreply.github.com>
2023-04-12 20:43:49 +02:00
Kwitsch 68a8476e48
chore(build): Automation workflows (#972)
* Added fork sync workflow(syncs the main branch with upstream every 30 minutes)
* Added monthly workflow run deletion(all skipped or canceld and runs older than 30days will be deleted)
* Removed Docker image build for main branch on forks
* Added documentation how to enable sync and docker build workflows
2023-04-03 16:33:16 +02:00
ThinkChaos f887e82fdc fix: support IDNA in lists
Not sure this is actually supported by most adblocker, but we might as
well be permissive.
2023-03-29 10:05:50 -04:00
Dimitri Herzog 3b9fd7bafe
refactor: use groupedCache to optimize cache access (#944)
* refactor: use groupedCache to optimize cache access

* refactor: fix review findings
2023-03-27 13:23:01 +02:00
ThinkChaos 5088c75a78
refactor: configuration rework (usage and printing) (#920)
* refactor: make `config.Duration` a struct with `time.Duration` embed

Allows directly calling `time.Duration` methods.

* refactor(HostsFileResolver): don't copy individual config items

The idea is to make adding configuration options easier, and searching
for references straight forward.

* refactor: move config printing to struct and use a logger

Using a logger allows using multiple levels so the whole configuration
can be printed in trace/verbose mode, but only important parts are
shown by default.

* squash: rename `Cast` to `ToDuration`

* squash: revert `Duration` to a simple wrapper ("new type" pattern)

* squash: `Duration.IsZero` tests

* squash: refactor resolvers to rely on their config directly if possible

* squash: implement `IsEnabled` and `LogValues` for all resolvers

* refactor: use go-enum `--values` to simplify getting all log fields

* refactor: simplify `QType` unmarshaling

* squash: rename `ValueLogger` to `Configurable`

* squash: rename `UpstreamConfig` to `ParallelBestConfig`

* squash: rename `RewriteConfig` to `RewriterConfig`

* squash: config tests

* squash: resolver tests

* squash: add `ForEach` test and improve `Chain` ones

* squash: simplify implementing `config.Configurable`

* squash: minor changes for better coverage

* squash: more `UnmarshalYAML` -> `UnmarshalText`

* refactor: move `config.Upstream` into own file

* refactor: add `Resolver.Type` method

* squash: add `log` method to `typed` to use `Resolover.Type` as prefix

* squash: tweak startup config logging

* squash: add `LogResolverConfig` tests

* squash: make sure all options of type `Duration` use `%s`
2023-03-12 22:14:10 +01:00
ThinkChaos a2ab7c3ef1
feat: allow hosts file resolver to use a HTTP(S) link or inline block (#884)
Unify the hosts file parsing between the hosts resolver and lists so
the resolver supports more data sources than local files.

Lists' group cache is now re-used if refresh fails.

Also improve lookups in hosts:
Instead of iterating through all hosts+aliases for each A/AAAA query,
we can do a single lookup.
For PTR we search through only the hosts with an IP version that matches
the question. And compare IPs instead of building the reverse DNS name
for each IP in the hosts database.
2023-03-06 19:32:41 -05:00
Shizun Ge c199750b0d
chore(log): always report group name when cache populating failed. (#875)
Co-authored-by: ThinkChaos <ThinkChaos@users.noreply.github.com>
2023-02-17 08:57:27 +01:00
ThinkChaos 6548d15d8b fix: ignore end of line comments when parsing lists 2023-02-07 09:35:53 -05:00
Kwitsch 9abeaeddea
Added gofumpt (#805)
* fix linter errors

* added gofumpt

* Makefile cleanup

* gofumpt run

* Update Makefile

Co-authored-by: ThinkChaos <ThinkChaos@users.noreply.github.com>

* go mod tidy

* fmt + lint

* go mod tidy

Co-authored-by: ThinkChaos <ThinkChaos@users.noreply.github.com>
2022-12-26 22:11:45 +01:00
Dimitri Herzog 3c40ce5d3b
fix: no initial cache refresh with "fast" start strategy (#804) 2022-12-17 23:06:58 +01:00
ThinkChaos d501d867df style(resolver): standardize configuration disabled impls and tests 2022-12-04 20:59:12 -05:00
Dimitri Herzog f78a57a94d
chore(build): add additional linters (#774)
* chore(build): add nolintlint linter

* chore(build): add usestdlibvars linter
2022-11-29 21:58:26 +01:00
Dimitri Herzog 3e95b12eed feat: caching for empty DNS responses (#700) 2022-11-08 21:45:28 +01:00
Kwitsch fd93f67899
feat: new parameter "StartStrategy" (#566) (#636)
Blocky should start resolving DNS traffic as soon as possible
2022-09-03 22:12:07 +02:00
FileGo 377f4764fe
Enable start as long as at least one upstream resolver in group is reachable (#608)
* Enable start if one upstream resolver fails

* Will now check if upstream actually works

* Fixed default upstream in some tests

* Increase timeouts in some tests

* change default value of "StartVerifyUpstream" to false

Co-authored-by: Dimitri Herzog <dimitri.herzog@gmail.com>
2022-08-21 17:21:08 +02:00
Dimitri Herzog 421807fc22
chore: update golangci-lint (#631) 2022-08-19 22:04:35 +02:00
Dimitri Herzog 3c5b7a20dc
chore: Managing of external tools with go modules (#616) (#617) 2022-08-03 22:10:07 +02:00
Dimitri Herzog b1973702bd chore: update golangci-lint to v1.47.3 2022-08-01 23:19:35 +02:00
Kwitsch 51ef9ea77f
test: fix flaky test(537) - second hunt (#610)
* go test -> ginkgo

* removed flake-attempts as suggested

* added FlakeAttempts to flaky test definition

* as net.DNSError qualifies as net.Error check and correct the response

* fix in downloader.go resolves flaky tests

* unwrap maybe?

* excluede DownloadFile from funlen linter

* use eventually on io operations in file_writer_test

* file_writer_test flaky fixes

* fix linter errors

* Serve test reversed go routine

* matcher fix

* serve test rework 2

* DeferCleanup tmp files

* fixed some flakiness in resolver tests

* go mod tidy

* fixed linter errors

* JustAfterEach -> DeferCleanup

* changed close to defer

* moved unwrap from downloader to test

* fix linter error

* propper cleanup in "should return error"

Co-authored-by: Dimitri Herzog <dimitri.herzog@gmail.com>
2022-08-01 22:27:11 +02:00
Kwitsch 061419f68f
chore: use ginkgo for test executions (#537) (#562)
* go test -> ginkgo

* added FlakeAttempts to flaky test definition

Co-authored-by: Dimitri Herzog <dimitri.herzog@gmail.com>
2022-07-08 08:04:08 +02:00
Dimitri Herzog 3adeea6691 chore: added debug output 2022-06-07 15:26:52 +02:00
FileGo 2944debf0b
Update to Go 1.18 (#540)
* Update to Go 1.18

* remove "nolint:unparam", since unparam is currently disabled with go 1.18

* update golangci-lint to 1.46.2

Co-authored-by: Dimitri Herzog <dimitri.herzog@gmail.com>
2022-05-27 22:41:11 +02:00
FileGo 8bb4f73a38
Removed net.Error.Temporary (#536) 2022-05-27 22:22:00 +02:00
Dimitri Herzog 3392de0e16
Cache refresh memory consumption optimization (#470) (#515) 2022-05-16 21:32:16 +02:00
Dimitri Herzog a4b89537db
update golangci-lint (#510)
* update golangci-lint

* enable gomnd linter

* enable asciicheck linter

* enable bidichk linter

* enable durationcheck linter

* enable errchkjson linter

* enable errorlint linter

* enable exhaustive linter

* enable gomoddirectives linter

* enable gomodguard guard

* enable grouper linter

* enable grouper and ifshort linters

* enable importas linter

* enable makezero linter

* enable nestif linter

* enable nilerr linter

* enable nilnil linter

* enable nlreturn linter

* enable nolintlint linter

* enable predeclared linter

* enable sqlclosecheck linter

* enable tenv linter

* enable wastedassign linter
2022-05-10 09:09:50 +02:00
Dimitri Herzog 53814a2208
chore: refactoring list cache, extracting download functionality (#508) 2022-05-06 17:57:33 +02:00
ThinkChaos 62c7816ab3
Support of DoH/DoT as bootstrap DNS (#485) 2022-04-22 22:12:35 +02:00
Dimitri Herzog 476dd7d0eb update go-enum to v0.4.0 2022-03-28 21:35:48 +02:00
ThinkChaos 27c8cbf2b7
Update to gingko v2 (#447) 2022-03-03 11:27:27 +01:00
Dimitri Herzog 759f55d89c
Rewrite DNS response cache (#378) (#413) 2022-01-20 16:45:29 +01:00
Dimitri Herzog 2b6c934c6b increased timeouts in test 2022-01-07 22:02:25 +01:00
FileGo 926f06ccdb
Add failed_download_count prometheus metric (#309) (#395)
Prometheus metric "failed_download_count" for failed downloads
2022-01-04 21:24:49 +01:00
Dimitri Herzog cdec99567a updated retry-go 2021-12-24 22:40:06 +01:00
Dimitri Herzog d7bf373b52 Replace all sleeps in tests with "eventually" from gomega (#379) 2021-12-22 22:21:04 +01:00
Dimitri Herzog 4b85eedc86 Replace retry implementation with retry-go 2021-12-13 22:10:14 +01:00
Kwitsch 2f79086742
Make retry for list download configurable (#308) (#338)
* DownloadAttempts & DownloadCooldown added to BlockingConfig

* implementation of downloadAttempts & downloadCooldown

* extended NewListCache call

* unit test fix(use old default values)

* documentation of downloadAttempts & downloadCooldown

* linter error(line length)
2021-11-14 21:34:09 +01:00
Dimitri Herzog 9b9596970c Improved configuration output (duration format, etc.) 2021-11-07 21:35:16 +01:00
Dimitri Herzog cd767960a5 introduced go-multierror to handle multiple errors 2021-10-13 22:45:32 +02:00
Dimitri Herzog 57036aa088 fixed golangci-lint issues 2021-10-13 21:40:18 +02:00
Kwitsch e5b44f49ca
application startup should fail if initial download of a single list failed (#310) (#313)
application startup should fail if initial download of a single list failed
2021-10-13 21:30:14 +02:00
Dimitri Herzog 138172c9c5 reduce the configuration output for inline defined lists 2021-09-19 22:50:43 +02:00
Dimitri Herzog ae9b12f15a Configurable upstream lookup timeout (#256) 2021-09-19 22:49:38 +02:00
Dimitri Herzog e7ddab714b regex support for matching (#12) 2021-09-18 22:51:20 +02:00