summaryrefslogtreecommitdiff
path: root/searx/network/client.py
Commit message (Collapse)AuthorAge
* Make it work with debian packages.HEADmasterAnton Khirnov2024-01-20
|
* [mod] searx.network: memory optimizationAlexandre Flament2023-08-27
| | | | | | | | Avoid to create a SSLContext in AsyncHTTPTransportNoHttp See: * https://github.com/encode/httpx/blob/0f61aa58d66680c239ce43c8cdd453e7dc532bfc/httpx/_transports/default.py#L271 * https://github.com/encode/httpx/issues/2298
* Bump httpx 0.21.2 from to 0.24.1Alexandre Flament2023-08-21
|
* [mod] Shuffle httpx's default ciphers of a SSL context randomly.Markus Heiser2023-03-19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | From the analyse of @9Ninety [1] we know that DDG (and may be other engines / I have startpage in mind) does some kind of TLS fingerprint to block bots. This patch shuffles the default ciphers from httpx to avoid a cipher profile that is known to httpx (and blocked by DDG). [1] https://github.com/searxng/searxng/issues/2246#issuecomment-1467895556 ---- From `What Is TLS Fingerprint and How to Bypass It`_ > When implementing TLS fingerprinting, servers can't operate based on a > locked-in whitelist database of fingerprints. New fingerprints appear > when web clients or TLS libraries release new versions. So, they have to > live off a blocklist database instead. > ... > It's safe to leave the first three as is but shuffle the remaining ciphers > and you can bypass the TLS fingerprint check. .. _What Is TLS Fingerprint and How to Bypass It: https://www.zenrows.com/blog/what-is-tls-fingerprint#how-to-bypass-tls-fingerprinting Signed-off-by: Markus Heiser <markus.heiser@darmarit.de> Closes: https://github.com/searxng/searxng/issues/2246
* searx.network: add "verify" option to the networksAlexandre Flament2022-10-14
| | | | | | | | | Each network can define a verify option: * false to disable certificate verification * a path to existing certificate. SearXNG uses SSL_CERT_FILE and SSL_CERT_DIR when they are defined see https://www.python-httpx.org/environment_variables/#ssl_cert_file
* [typing] add type hints for dictionariesMartin Fischer2022-01-17
|
* [mod] upgrade httpx 0.21.2Alexandre Flament2022-01-05
| | | | | | | | httpx 0.21.2 and httpcore 0.14.4 fix multiple issues: * https://github.com/encode/httpx/releases/tag/0.21.2 * https://github.com/encode/httpcore/releases/tag/0.14.4 so most of the workarounds in searx.network have been removed.
* [format.python] initial formatting of the python codeMarkus Heiser2021-12-27
| | | | | | | | | | This patch was generated by black [1]:: make format.python [1] https://github.com/psf/black Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
* [fix] searx.network.stream: fix memory leakAlexandre Flament2021-09-28
|
* Merge pull request #261 from dalf/upgrade_httpxAlexandre Flament2021-09-17
|\ | | | | [upd] upgrade httpx 0.19.0
| * [mod] searx.network: remove redundant codeAlexandre Flament2021-09-17
| | | | | | | | | | | | searx.client.new_client: the proxies parameter is a dictonnary, and the protocol (key of the dictionnary) is already normalized (see usage of searx.network.network.PROXY_PATTERN_MAPPING)
| * [upd] upgrade httpx 0.19.0Alexandre Flament2021-09-17
| | | | | | | | | | adjust searx.network module to the new internal API see https://github.com/encode/httpx/pull/1522
* | [pylint] fix global-variable-not-assigned issuesMarkus Heiser2021-09-17
|/ | | | | | | | | | | | | | | | | | | If there is no write access, there is no need for global. Remove global statement if there is no assignment. global-variable-not-assigned: Using global for names but no assignment is done Used when a variable is defined through the "global" statement but no assignment to this variable is done. In Pylint 2.11 the global-variable-not-assigned checker now catches global variables that are never reassigned in a local scope and catches (reassigned) functions [1][2] [1] https://pylint.pycqa.org/en/latest/whatsnew/2.11.html [2] https://github.com/PyCQA/pylint/issues/1375 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
* [mod] debug mode: log HTTP requests with network nameAlexandre Flament2021-09-11
| | | | | | For example wikipedia requests use the logger name "searx.network.wikipedia" Log is disable when searx_debug is False
* [pylint] searx: drop no longer needed 'missing-function-docstring'Markus Heiser2021-09-07
| | | | | Suggested-by: @dalf https://github.com/searxng/searxng/issues/102#issuecomment-914168470 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
* [coding-style] searx/network/client.py - normalized indentationsMarkus Heiser2021-05-24
| | | | | | No functional change! Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
* [pylint] searx/network/client.py & add global (TRANSPORT_KWARGS)Markus Heiser2021-05-24
| | | | | | | | | | No functional change! - fix messages from pylint - add ``global TRANSPORT_KWARGS`` - normalized python_socks imports Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
* [mod] searx.network.client: the same configuration reuses the same ↵Alexandre Flament2021-05-05
| | | | | | | | ssl.SSLContext before there was one ssl.SSLContext per client. see https://github.com/encode/httpx/issues/978
* [fix] searx.network: fix rare cases where LOOP is NoneAlexandre Flament2021-04-27
| | | | | | | | | * searx.network.client.LOOP is initialized in a thread * searx.network.__init__ imports LOOP which may happen before the thread has initialized LOOP This commit adds a new function "searx.network.client.get_loop()" to fix this issue
* [httpx] replace searx.poolrequests by searx.networkAlexandre Flament2021-04-12
settings.yml: * outgoing.networks: * can contains network definition * propertiers: enable_http, verify, http2, max_connections, max_keepalive_connections, keepalive_expiry, local_addresses, support_ipv4, support_ipv6, proxies, max_redirects, retries * retries: 0 by default, number of times searx retries to send the HTTP request (using different IP & proxy each time) * local_addresses can be "192.168.0.1/24" (it supports IPv6) * support_ipv4 & support_ipv6: both True by default see https://github.com/searx/searx/pull/1034 * each engine can define a "network" section: * either a full network description * either reference an existing network * all HTTP requests of engine use the same HTTP configuration (it was not the case before, see proxy configuration in master)
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-18 23:32:51 +0000