Commit Graph

293 Commits

Author SHA1 Message Date
Dimitri Herzog 4a5a395655
fix: log.privacy option no longer hides domain names being resolved (#1255) 2023-11-17 15:59:22 +01:00
Kwitsch 8ece708fe9
devcontainer & unit test refactoring (#1220)
* extension cleanup & added ginkgo watch

* added gcov2lcov

* added test explorer and reworked scripts

* go mod tidy

* use package cache volume

* script rework

* defined tasks

* defined launch

* don't try to convert if test was canceld

* generate lcov only in devcontainer

* disable coverage upload on forks

* wip: make lcov

* fixed unit tests for parallel

* parallel test for lists

* fix serve test for parallel

* parallel test fixes

* deleted accident commit

* wip: make lcov

* restructured settings location

* start script refactoring

* added GetProcessPort

* fixed parallel ports

* race fix

* changed port for github runner binding

* fixed local list var in test

* more local vars in tests fix

* less local vars

* run test & race parallel

* removed invalid error check

* fixed error  check

* less local variables

* fixed timing problem

* removed gcov2lcov

* added generate-lcov

* added GINKGO_PROCS to makefile

* fixed workflow

* run generate-lcov on save *.go

* added tooltitude
2023-11-11 11:58:31 -05:00
Dimitri Herzog 73bc60cb22
feat: reduce DNS response cache memory consumption (#1196)
---------

Co-authored-by: ThinkChaos <ThinkChaos@users.noreply.github.com>
2023-10-10 17:56:11 -04:00
Dimitri Herzog 33ea933015
refactor: pass context for goroutine shutdown (#1187) 2023-10-07 16:21:40 -04:00
Dimitri Herzog d77f0ed54f
feat: add API endpoint to flush the DNS Cache (#1178) 2023-09-30 16:13:01 -04:00
Dimitri Herzog 96e812d57e
chore: remove unnecessary lock (#1177)
---------

Co-authored-by: ThinkChaos <ThinkChaos@users.noreply.github.com>
2023-09-30 21:19:47 +02:00
Dimitri Herzog 497bd0d0fd
chore(refactor): refactor cache implementation (#1174)
* chore(refactor): refactor cache implementation

* chore: use atomic.Uint32 as prefetch names query count

Co-authored-by: ThinkChaos <ThinkChaos@users.noreply.github.com>

---------

Co-authored-by: ThinkChaos <ThinkChaos@users.noreply.github.com>
2023-09-30 17:14:59 +02:00
Dimitri Herzog f98859325e fix: use different TTL of multiple records in answer 2023-09-25 22:12:25 +02:00
Dimitri Herzog 431b9be3e5 fix: don't cache DNS responses with CD flag 2023-09-20 23:16:49 +02:00
Dimitri Herzog 6f60bea5c2
feat: add timeout to bootstrap (#1158) 2023-09-20 16:41:55 -04:00
Dimitri Herzog 500c187082 chore(test): fix race conditions 2023-09-18 18:46:01 +02:00
Dimitri Herzog 9f15228c09
fix: don't cache EDNS pseudo records (#1150)
Co-authored-by: ThinkChaos <ThinkChaos@users.noreply.github.com>
2023-09-18 09:04:08 +02:00
Dimitri Herzog d5be5884d9
fix: truncated responses should not be cached (#1149) 2023-09-17 09:43:29 +02:00
Dimitri Herzog 47eb8be77f
fix: conditional resolver panics on response without question (#1148) 2023-09-16 11:51:32 +02:00
Dimitri Herzog ed864962b2 chore: improve execution time of e2e tests 2023-09-14 15:06:10 +02:00
Dimitri Herzog b1d014017b feat(querylog): add flushInterval parameter 2023-09-14 15:06:10 +02:00
Dimitri Herzog 72d747c16f
feat: API-first approach for REST interface (#1129)
* feat: embed OpenAPI definition file

* feat: use OpenAPI generated server and client

* feat: provide OpenAPI interface documentation

* chore(test): add additional tests
2023-09-09 19:30:55 +02:00
Dimitri Herzog 7645ed35a3
chore(build): update golangci-lint to 1.54.2 (#1121) 2023-08-31 14:47:30 +02:00
DerRockWolf c112e86740
feat: add upstream strategy `strict` (#1093) 2023-08-21 09:50:23 +02:00
ThinkChaos 639fba5323 refactor(config): allow more configuration for `upstreams`
Rename the `upstream` option to `upstreams.groups` so we can have
more `upstreams` options.
2023-08-01 15:01:40 -04:00
DerRockWolf dc9bba59a1
fix: use correct resolver in "using response from resolver" debug log (#1082) 2023-07-27 12:08:36 -04:00
ThinkChaos 56633da0a7 feat(sudn): refactor and expand supported special use domains RFCs 2023-07-14 17:38:13 -04:00
ThinkChaos 8a93e4500c style: fix all existing lint errors 2023-07-07 09:16:20 -04:00
ThinkChaos f22e310501 fix: duration checks to take into account values can be negative
Replace `IsZero` with `IsAboveZero` to help us avoid this mistake again.
2023-07-07 09:16:20 -04:00
ThinkChaos cfc3699ab5 feat: support multiple hosts files 2023-07-07 09:16:20 -04:00
mandrav 015b565137
Add maxErrorsPerFile blocking configuration (#986)
* Add maxErrorsPerFile blocking configuration

The default max errors per file of 5 is too small IMHO.
This commit makes this number user-configurable.

* squash: fix lint

* squash: docs

* squash: change type to int to allow -1

* squash: test that the `maxErrorsPerFile` is actually used

---------

Co-authored-by: ThinkChaos <ThinkChaos@users.noreply.github.com>
2023-04-12 20:43:49 +02:00
Dimitri Herzog 3b9fd7bafe
refactor: use groupedCache to optimize cache access (#944)
* refactor: use groupedCache to optimize cache access

* refactor: fix review findings
2023-03-27 13:23:01 +02:00
Dimitri Herzog b44e4146ab
chore(build): update weightedrand to v2 (#934) 2023-03-16 07:49:23 +01:00
Dimitri Herzog 0157f4ce56
refactor: add generics to ExpiringCache (#933) 2023-03-15 20:32:35 +01:00
Dimitri Herzog db1e2fd655
chore(test): fix freezing test (#932) 2023-03-15 20:25:52 +01:00
ThinkChaos 5088c75a78
refactor: configuration rework (usage and printing) (#920)
* refactor: make `config.Duration` a struct with `time.Duration` embed

Allows directly calling `time.Duration` methods.

* refactor(HostsFileResolver): don't copy individual config items

The idea is to make adding configuration options easier, and searching
for references straight forward.

* refactor: move config printing to struct and use a logger

Using a logger allows using multiple levels so the whole configuration
can be printed in trace/verbose mode, but only important parts are
shown by default.

* squash: rename `Cast` to `ToDuration`

* squash: revert `Duration` to a simple wrapper ("new type" pattern)

* squash: `Duration.IsZero` tests

* squash: refactor resolvers to rely on their config directly if possible

* squash: implement `IsEnabled` and `LogValues` for all resolvers

* refactor: use go-enum `--values` to simplify getting all log fields

* refactor: simplify `QType` unmarshaling

* squash: rename `ValueLogger` to `Configurable`

* squash: rename `UpstreamConfig` to `ParallelBestConfig`

* squash: rename `RewriteConfig` to `RewriterConfig`

* squash: config tests

* squash: resolver tests

* squash: add `ForEach` test and improve `Chain` ones

* squash: simplify implementing `config.Configurable`

* squash: minor changes for better coverage

* squash: more `UnmarshalYAML` -> `UnmarshalText`

* refactor: move `config.Upstream` into own file

* refactor: add `Resolver.Type` method

* squash: add `log` method to `typed` to use `Resolover.Type` as prefix

* squash: tweak startup config logging

* squash: add `LogResolverConfig` tests

* squash: make sure all options of type `Duration` use `%s`
2023-03-12 22:14:10 +01:00
Dimitri Herzog f2e488678b
fix: wrong prefetch metrics if bootstrap is enabled (#891) (#915) 2023-03-07 16:04:27 +01:00
Dimitri Herzog 120e32c1eb
feat: client name in clientGroupsBlock should not be case-sensitive (#894) (#913) 2023-03-07 14:23:02 +01:00
ThinkChaos a2ab7c3ef1
feat: allow hosts file resolver to use a HTTP(S) link or inline block (#884)
Unify the hosts file parsing between the hosts resolver and lists so
the resolver supports more data sources than local files.

Lists' group cache is now re-used if refresh fails.

Also improve lookups in hosts:
Instead of iterating through all hosts+aliases for each A/AAAA query,
we can do a single lookup.
For PTR we search through only the hosts with an IP version that matches
the question. And compare IPs instead of building the reverse DNS name
for each IP in the hosts database.
2023-03-06 19:32:41 -05:00
Dimitri Herzog 7da4724c03
feat: add cache entire DNS response (#833) (#909) 2023-03-03 21:39:44 +01:00
ThinkChaos de9acd3849
feat: add upstream string to resolve error (#902) 2023-02-28 16:22:27 +01:00
Kwitsch 101e2c554b
Workflow changes (#857)
* fix linter errors

* fix for flaky redis unit test

* timeout adjustment

* e2e-test -> ci-build

* setup docker buildx

* added concurrency cancelation

* changed job name

* Test comment out problematic eventually

* added comment

* workflow file rename

* workflow name changed

* skip go generate

* added make generate

* removed unused go generate

* setup golang with caches if needed

* matrix revamp

* activate gofor e2e

* fix matrix

* test

* eventually timing

* removed linter workflow in favor of make matrix

* expect -> Eventually

* renamed workflow
2023-02-07 14:00:53 +01:00
ThinkChaos 012c8d49f8 test(bootstrap): add connectIPVersion tests for HTTP Transport 2023-01-23 19:30:02 -05:00
ThinkChaos 2cb826db22 fix: `connectIPVersion` not being respected for upstream connections 2023-01-23 19:30:02 -05:00
ThinkChaos 63f65002e8 feat: always prefetch upstream IPs to avoid stalling user queries
Otherwise, a request to blocky could end up waiting for 2 DNS requests:
  1. lookup the DNS server IP
  2. forward the user request to the server looked-up in 1
2023-01-18 18:57:55 +01:00
ThinkChaos 7c76836373 feat: stack log prefixes to differentiate between log emitters
The goal is to be able to tell apart logs from difference sources, such
as `bootstrap.parallel_best_resolver` and `parallel_best_resolver`.
2023-01-18 18:57:55 +01:00
ThinkChaos e47a8b7f60 feat(bootstrap): support IP only encrypted DNS
Also make `tcp+udp` upstreams use any IPs provided.
2023-01-18 18:57:55 +01:00
ThinkChaos a79459987b feat(bootstrap): support multiple upstreams
If more than one upstream is configured, they are raced via
a `ParallelBestResolver`.
2023-01-18 18:57:55 +01:00
Dimitri Herzog 53a7d4fccc
Test refactoring (#798)
* test: refactor tests

* chore: fix possible race condition in cache
2022-12-29 14:58:25 +01:00
Kwitsch 9abeaeddea
Added gofumpt (#805)
* fix linter errors

* added gofumpt

* Makefile cleanup

* gofumpt run

* Update Makefile

Co-authored-by: ThinkChaos <ThinkChaos@users.noreply.github.com>

* go mod tidy

* fmt + lint

* go mod tidy

Co-authored-by: ThinkChaos <ThinkChaos@users.noreply.github.com>
2022-12-26 22:11:45 +01:00
Dimitri Herzog 3c40ce5d3b
fix: no initial cache refresh with "fast" start strategy (#804) 2022-12-17 23:06:58 +01:00
Dimitri Herzog 87779d5839
fix(resolver): crash when using client upstream (#791) (#793) 2022-12-09 18:42:06 +01:00
ThinkChaos 4f01d017df fix(resolver): query log creation cooldown not being fixed
Default is backoff, but documentation implies a fixed delay.
2022-12-09 11:23:28 -05:00
ThinkChaos 164e398d04 fix(resolver): upstream long retry delay
Default delay is 100ms which is way too much for query retries.
2022-12-09 11:23:28 -05:00
ThinkChaos d501d867df style(resolver): standardize configuration disabled impls and tests 2022-12-04 20:59:12 -05:00
ThinkChaos 7fa7e2ec79 refactor(resolver): only build mocks during tests
This makes iterating with breaking refactors easier as you can also
break the mocks without breaking normal compilation.

Also unexport code only used in the resolver package.
Only `MockUDPUpstreamServer` is kept as non test and exported as it is
used by the server package.
2022-12-04 20:58:59 -05:00
ThinkChaos c06c017a1a fix: `startVerifyUpstream` not disabling all start checks 2022-12-02 12:56:44 -05:00
ThinkChaos add591c5a4
fix(ede-resolver): don't panic when next resolver errors (#778) 2022-12-02 10:10:11 +01:00
Dimitri Herzog f78a57a94d
chore(build): add additional linters (#774)
* chore(build): add nolintlint linter

* chore(build): add usestdlibvars linter
2022-11-29 21:58:26 +01:00
Dimitri Herzog 2b49c2048f
feat(querylog): define which information should be logged (#765) (#766) 2022-11-26 22:12:56 +01:00
Dimitri Herzog 3e95b12eed feat: caching for empty DNS responses (#700) 2022-11-08 21:45:28 +01:00
Dimitri Herzog d8c36063b0 fix: format ipv6 URL for DoH bootstrap request according to RFC3513 (#701) 2022-11-04 08:29:16 +01:00
Kwitsch fdb09bb902 more flaky fixes 2022-09-24 00:02:58 +02:00
Kwitsch 3dc6cb33f9 flaky fixes 2022-09-23 23:42:18 +02:00
Kwitsch 0eadfc6bf6 fix flaky race test 2022-09-23 23:26:08 +02:00
Kwitsch d93ed9c3c7 refactoring 2022-09-23 23:07:54 +02:00
Kwitsch 93e1e32582 Merge branch 'fb-635' of https://github.com/kwitsch/blocky-dev into fb-635
# Conflicts:
#	resolver/sudn_resolver.go
2022-09-23 22:58:35 +02:00
Kwitsch 9784134ab7 moved newResponseMsg & returnResponseModel to resolver.go 2022-09-23 22:56:40 +02:00
Kwitsch ebe284dbdd
Merge branch 'development' into fb-635 2022-09-20 21:01:48 +02:00
ThinkChaos f48908f7be
Add connectIPVersion to choose how Blocky connects to other hosts (#589)
This impacts both DNS lookups and lists downloading.

Co-authored-by: Dimitri Herzog <dimitri.herzog@gmail.com>
2022-09-19 21:44:12 +02:00
Kwitsch fa75c9d44b
fixed typo
Co-authored-by: ThinkChaos <ThinkChaos@users.noreply.github.com>
2022-09-18 14:26:01 +02:00
Dan Ponte 81344f714c Address code review concerns 2022-09-17 15:34:34 -04:00
Dan Ponte d889419f67 No intermediate 2022-09-15 13:09:50 -04:00
Dan Ponte 46deb2f780 Add ability to specify certificate common name. 2022-09-14 22:03:15 -04:00
Kwitsch ba8fe266d5 mdns leakage block 2022-09-14 08:44:17 +02:00
Kwitsch 4f4d97c2eb added RFC6762 & did some refactoring 2022-09-13 19:56:50 +02:00
Kwitsch 2da657f1da linter 2022-09-10 14:10:51 +02:00
Kwitsch cf84241e48 added IPv4loopback var 2022-09-10 04:05:14 +02:00
Kwitsch e3b6810b8b
use IPv6loopback instead reparsing it
Co-authored-by: ThinkChaos <ThinkChaos@users.noreply.github.com>
2022-09-10 04:01:07 +02:00
Kwitsch b50d7c664f
removed debugging code
Co-authored-by: ThinkChaos <ThinkChaos@users.noreply.github.com>
2022-09-10 03:56:52 +02:00
Kwitsch e65a615418 Merge branch 'development' into fb-635 2022-09-09 21:11:15 +02:00
Kwitsch 2f7a30a3b9 rename SudnResolver to SpecialUseDomainNamesResolver 2022-09-09 21:02:01 +02:00
Kwitsch 156f068ace removed configuration output 2022-09-06 21:45:08 +02:00
Kwitsch d809b680aa refactoring 2022-09-06 21:33:48 +02:00
Kwitsch 0d225114ae added SetReply 2022-09-06 21:22:17 +02:00
Kwitsch a0453a733b added unit tests 2022-09-04 01:27:24 +02:00
Kwitsch 6bb3c060e8 finish sudn resolver 2022-09-04 01:27:13 +02:00
Dimitri Herzog 48dbc5bc13 update: github.com/onsi/gomega 2022-09-03 23:43:26 +02:00
Kwitsch fd93f67899
feat: new parameter "StartStrategy" (#566) (#636)
Blocky should start resolving DNS traffic as soon as possible
2022-09-03 22:12:07 +02:00
Kwitsch f303d9cab1 sudnLocalhost 2022-09-02 19:52:16 +02:00
Kwitsch baf221cf44 added SudnResolver 2022-08-27 00:00:48 +02:00
Kwitsch 2596f922f5 added ResponseTypeSPECIAL to convertToExtendedErrorCode 2022-08-26 23:59:46 +02:00
FileGo b49695fad3 Added config, simplified check for loopback 2022-08-23 07:54:03 +01:00
FileGo 7ce875f105 Filter loopback addresses from hosts file 2022-08-22 09:28:55 +01:00
FileGo 377f4764fe
Enable start as long as at least one upstream resolver in group is reachable (#608)
* Enable start if one upstream resolver fails

* Will now check if upstream actually works

* Fixed default upstream in some tests

* Increase timeouts in some tests

* change default value of "StartVerifyUpstream" to false

Co-authored-by: Dimitri Herzog <dimitri.herzog@gmail.com>
2022-08-21 17:21:08 +02:00
Dimitri Herzog 421807fc22
chore: update golangci-lint (#631) 2022-08-19 22:04:35 +02:00
Kwitsch 9c80a5f9a9
test: use dynamic unit test data (#624) 2022-08-06 22:44:26 +02:00
Kwitsch 5e9eaa2965
delivery of blocking group as EDNS error(560) (#611) 2022-08-06 22:30:26 +02:00
Kwitsch 51ef9ea77f
test: fix flaky test(537) - second hunt (#610)
* go test -> ginkgo

* removed flake-attempts as suggested

* added FlakeAttempts to flaky test definition

* as net.DNSError qualifies as net.Error check and correct the response

* fix in downloader.go resolves flaky tests

* unwrap maybe?

* excluede DownloadFile from funlen linter

* use eventually on io operations in file_writer_test

* file_writer_test flaky fixes

* fix linter errors

* Serve test reversed go routine

* matcher fix

* serve test rework 2

* DeferCleanup tmp files

* fixed some flakiness in resolver tests

* go mod tidy

* fixed linter errors

* JustAfterEach -> DeferCleanup

* changed close to defer

* moved unwrap from downloader to test

* fix linter error

* propper cleanup in "should return error"

Co-authored-by: Dimitri Herzog <dimitri.herzog@gmail.com>
2022-08-01 22:27:11 +02:00
Adyanth H 31cf8c77d5
Add fallbackUpstream for conditional mapping (#593) 2022-07-11 08:06:42 +02:00
Dimitri Herzog 3b620102a7
set host header for DoH requests (#580) 2022-06-29 21:39:39 +02:00
Kwitsch c912356740
Option to handle FQDN only requests (#561) 2022-06-20 13:02:51 +02:00
Dimitri Herzog fd8c61c57c improved LRU cache operations (avoid write lock on TotalCount) 2022-06-02 22:58:52 +02:00
FileGo 8bb4f73a38
Removed net.Error.Temporary (#536) 2022-05-27 22:22:00 +02:00
FileGo 776c51fc59
Adds user customizable DoH upstream user agent (#525)
* Adds user customizable DoH upstream user agent

* Changed default user agent to empty
2022-05-18 08:49:15 +02:00