Commit Graph

55 Commits

Author SHA1 Message Date
ThinkChaos 8c7b89cbb3 refactor(tests): remove `TmpFile.Error` to centralize error checks 2023-12-01 15:53:11 -05:00
ThinkChaos 891d0fba74 refactor(tests): move `DeferCleanup` into helper functions 2023-12-01 15:53:11 -05:00
ThinkChaos f371857150 style(tests): use `ContainElements` where possible 2023-12-01 15:53:11 -05:00
Kwitsch 3378316982
Refactoring: FileDownloader (#1281)
* enabled containedctx & contextcheck

* enabled noctx

* less background context

* context metrics test

* use ginkgo context instead of background

* fix redis e2e tests

* made downloader context aware
2023-11-29 18:18:29 +01:00
Kwitsch 6c000090b1
Unit test refactoring & devcontainer enhancement(#1245)
* Whietlist dot imports for test packages

* improved generate-lcov script

* moved mock_call_sequence to helpertest

* removed default config for markdown

* generate seperate lcov.info for each directory

* path fix in lcov generation script

* ginkgo -> v2.13.1

* fixed redundant strings

* added exampleComKey

* added folder name to finish message

* hide lcov.info in devcontainer

* coverage plugin is bugged if lcov file is excluded

* moved all lcov files to coverage folder

* added requested toos #1251
2023-11-15 16:42:53 +01:00
Kwitsch 8ece708fe9
devcontainer & unit test refactoring (#1220)
* extension cleanup & added ginkgo watch

* added gcov2lcov

* added test explorer and reworked scripts

* go mod tidy

* use package cache volume

* script rework

* defined tasks

* defined launch

* don't try to convert if test was canceld

* generate lcov only in devcontainer

* disable coverage upload on forks

* wip: make lcov

* fixed unit tests for parallel

* parallel test for lists

* fix serve test for parallel

* parallel test fixes

* deleted accident commit

* wip: make lcov

* restructured settings location

* start script refactoring

* added GetProcessPort

* fixed parallel ports

* race fix

* changed port for github runner binding

* fixed local list var in test

* more local vars in tests fix

* less local vars

* run test & race parallel

* removed invalid error check

* fixed error  check

* less local variables

* fixed timing problem

* removed gcov2lcov

* added generate-lcov

* added GINKGO_PROCS to makefile

* fixed workflow

* run generate-lcov on save *.go

* added tooltitude
2023-11-11 11:58:31 -05:00
Dimitri Herzog 33ea933015
refactor: pass context for goroutine shutdown (#1187) 2023-10-07 16:21:40 -04:00
Dimitri Herzog 720e6550be chore(test): fix flaky test 2023-09-14 15:03:37 +02:00
Dimitri Herzog 72d747c16f
feat: API-first approach for REST interface (#1129)
* feat: embed OpenAPI definition file

* feat: use OpenAPI generated server and client

* feat: provide OpenAPI interface documentation

* chore(test): add additional tests
2023-09-09 19:30:55 +02:00
ThinkChaos cfc3699ab5 feat: support multiple hosts files 2023-07-07 09:16:20 -04:00
ThinkChaos 8253075f28
test: remove focus in list_cache suite (#990)
Accidentally left `FIt` when committing the test.
2023-04-13 09:10:40 +02:00
mandrav 015b565137
Add maxErrorsPerFile blocking configuration (#986)
* Add maxErrorsPerFile blocking configuration

The default max errors per file of 5 is too small IMHO.
This commit makes this number user-configurable.

* squash: fix lint

* squash: docs

* squash: change type to int to allow -1

* squash: test that the `maxErrorsPerFile` is actually used

---------

Co-authored-by: ThinkChaos <ThinkChaos@users.noreply.github.com>
2023-04-12 20:43:49 +02:00
Dimitri Herzog 3b9fd7bafe
refactor: use groupedCache to optimize cache access (#944)
* refactor: use groupedCache to optimize cache access

* refactor: fix review findings
2023-03-27 13:23:01 +02:00
ThinkChaos 5088c75a78
refactor: configuration rework (usage and printing) (#920)
* refactor: make `config.Duration` a struct with `time.Duration` embed

Allows directly calling `time.Duration` methods.

* refactor(HostsFileResolver): don't copy individual config items

The idea is to make adding configuration options easier, and searching
for references straight forward.

* refactor: move config printing to struct and use a logger

Using a logger allows using multiple levels so the whole configuration
can be printed in trace/verbose mode, but only important parts are
shown by default.

* squash: rename `Cast` to `ToDuration`

* squash: revert `Duration` to a simple wrapper ("new type" pattern)

* squash: `Duration.IsZero` tests

* squash: refactor resolvers to rely on their config directly if possible

* squash: implement `IsEnabled` and `LogValues` for all resolvers

* refactor: use go-enum `--values` to simplify getting all log fields

* refactor: simplify `QType` unmarshaling

* squash: rename `ValueLogger` to `Configurable`

* squash: rename `UpstreamConfig` to `ParallelBestConfig`

* squash: rename `RewriteConfig` to `RewriterConfig`

* squash: config tests

* squash: resolver tests

* squash: add `ForEach` test and improve `Chain` ones

* squash: simplify implementing `config.Configurable`

* squash: minor changes for better coverage

* squash: more `UnmarshalYAML` -> `UnmarshalText`

* refactor: move `config.Upstream` into own file

* refactor: add `Resolver.Type` method

* squash: add `log` method to `typed` to use `Resolover.Type` as prefix

* squash: tweak startup config logging

* squash: add `LogResolverConfig` tests

* squash: make sure all options of type `Duration` use `%s`
2023-03-12 22:14:10 +01:00
ThinkChaos a2ab7c3ef1
feat: allow hosts file resolver to use a HTTP(S) link or inline block (#884)
Unify the hosts file parsing between the hosts resolver and lists so
the resolver supports more data sources than local files.

Lists' group cache is now re-used if refresh fails.

Also improve lookups in hosts:
Instead of iterating through all hosts+aliases for each A/AAAA query,
we can do a single lookup.
For PTR we search through only the hosts with an IP version that matches
the question. And compare IPs instead of building the reverse DNS name
for each IP in the hosts database.
2023-03-06 19:32:41 -05:00
ThinkChaos 6548d15d8b fix: ignore end of line comments when parsing lists 2023-02-07 09:35:53 -05:00
Kwitsch 9abeaeddea
Added gofumpt (#805)
* fix linter errors

* added gofumpt

* Makefile cleanup

* gofumpt run

* Update Makefile

Co-authored-by: ThinkChaos <ThinkChaos@users.noreply.github.com>

* go mod tidy

* fmt + lint

* go mod tidy

Co-authored-by: ThinkChaos <ThinkChaos@users.noreply.github.com>
2022-12-26 22:11:45 +01:00
ThinkChaos d501d867df style(resolver): standardize configuration disabled impls and tests 2022-12-04 20:59:12 -05:00
Dimitri Herzog f78a57a94d
chore(build): add additional linters (#774)
* chore(build): add nolintlint linter

* chore(build): add usestdlibvars linter
2022-11-29 21:58:26 +01:00
Kwitsch fd93f67899
feat: new parameter "StartStrategy" (#566) (#636)
Blocky should start resolving DNS traffic as soon as possible
2022-09-03 22:12:07 +02:00
Dimitri Herzog 421807fc22
chore: update golangci-lint (#631) 2022-08-19 22:04:35 +02:00
Dimitri Herzog b1973702bd chore: update golangci-lint to v1.47.3 2022-08-01 23:19:35 +02:00
FileGo 2944debf0b
Update to Go 1.18 (#540)
* Update to Go 1.18

* remove "nolint:unparam", since unparam is currently disabled with go 1.18

* update golangci-lint to 1.46.2

Co-authored-by: Dimitri Herzog <dimitri.herzog@gmail.com>
2022-05-27 22:41:11 +02:00
Dimitri Herzog 3392de0e16
Cache refresh memory consumption optimization (#470) (#515) 2022-05-16 21:32:16 +02:00
Dimitri Herzog a4b89537db
update golangci-lint (#510)
* update golangci-lint

* enable gomnd linter

* enable asciicheck linter

* enable bidichk linter

* enable durationcheck linter

* enable errchkjson linter

* enable errorlint linter

* enable exhaustive linter

* enable gomoddirectives linter

* enable gomodguard guard

* enable grouper linter

* enable grouper and ifshort linters

* enable importas linter

* enable makezero linter

* enable nestif linter

* enable nilerr linter

* enable nilnil linter

* enable nlreturn linter

* enable nolintlint linter

* enable predeclared linter

* enable sqlclosecheck linter

* enable tenv linter

* enable wastedassign linter
2022-05-10 09:09:50 +02:00
Dimitri Herzog 53814a2208
chore: refactoring list cache, extracting download functionality (#508) 2022-05-06 17:57:33 +02:00
ThinkChaos 62c7816ab3
Support of DoH/DoT as bootstrap DNS (#485) 2022-04-22 22:12:35 +02:00
ThinkChaos 27c8cbf2b7
Update to gingko v2 (#447) 2022-03-03 11:27:27 +01:00
Dimitri Herzog 2b6c934c6b increased timeouts in test 2022-01-07 22:02:25 +01:00
FileGo 926f06ccdb
Add failed_download_count prometheus metric (#309) (#395)
Prometheus metric "failed_download_count" for failed downloads
2022-01-04 21:24:49 +01:00
Dimitri Herzog d7bf373b52 Replace all sleeps in tests with "eventually" from gomega (#379) 2021-12-22 22:21:04 +01:00
Kwitsch 2f79086742
Make retry for list download configurable (#308) (#338)
* DownloadAttempts & DownloadCooldown added to BlockingConfig

* implementation of downloadAttempts & downloadCooldown

* extended NewListCache call

* unit test fix(use old default values)

* documentation of downloadAttempts & downloadCooldown

* linter error(line length)
2021-11-14 21:34:09 +01:00
Dimitri Herzog cd767960a5 introduced go-multierror to handle multiple errors 2021-10-13 22:45:32 +02:00
Kwitsch e5b44f49ca
application startup should fail if initial download of a single list failed (#310) (#313)
application startup should fail if initial download of a single list failed
2021-10-13 21:30:14 +02:00
Dimitri Herzog 138172c9c5 reduce the configuration output for inline defined lists 2021-09-19 22:50:43 +02:00
Dimitri Herzog ae9b12f15a Configurable upstream lookup timeout (#256) 2021-09-19 22:49:38 +02:00
Dimitri Herzog e7ddab714b regex support for matching (#12) 2021-09-18 22:51:20 +02:00
Dimitri Herzog 925c6f97eb Make list download timeout configurable (#254) 2021-09-15 07:41:20 +02:00
Dimitri Herzog ee8f041938 refactoring: generate enums 2021-09-11 21:31:03 +02:00
Dimitri Herzog 8a2a3fb666 inline lists definition (#207) 2021-08-29 15:32:40 +02:00
Dimitri Herzog 919512959b use full-qualified name as module 2021-08-27 13:41:36 +02:00
moyo 30bde1614c
fix panic with empty query (#175) 2021-04-10 21:48:38 +02:00
Dimitri Herzog b043849159 command package refactoring 2021-02-08 22:00:11 +01:00
Dimitri Herzog 0fd25207d1 introducing event bus, refactorings 2021-01-23 22:29:43 +01:00
Dimitri Herzog 1d04fb065d #46: List download: Improve error handling 2020-05-11 22:18:29 +02:00
Dimitri Herzog a64d7461a3 #46: List download: Improve error handling 2020-05-10 19:48:31 +02:00
Dimitri Herzog 0ec6c8af9c Test rewrite (Ginkgo), Refactorings 2020-05-04 22:20:13 +02:00
Dimitri Herzog c6532d7939 #31: Fine grained locking on list update 2020-04-22 22:31:37 +02:00
Dimitri Herzog 0766c6480a #27: CLI tool 2020-04-08 23:03:07 +02:00
Dimitri Herzog acd63995fd added tests 2020-03-07 16:52:28 +01:00