Commit Graph

Select branches

Hide Pull Requests

main

Mono Color

• 3eaa7ffc25 ops: exclude site directory from git Karim shoair 2026-02-08 20:42:35 +02:00
• b1ce0b414b ops: Add mypy and pyright to the code quality workflow Karim shoair 2026-02-07 17:28:44 +02:00
• 16852cb1ac style: Use shorter and more accurate naming for constants Karim shoair 2026-02-07 16:37:47 +02:00
• ec7c7f8153 Merge branch 'main' into v4 Karim shoair 2026-02-07 16:30:37 +02:00
• e777342741 build: pump up version and deps Karim shoair 2026-02-07 16:30:32 +02:00
• 5ec929435b style: Fix all mypy errors and add type hints to untyped function bodies Karim shoair 2026-02-07 16:30:00 +02:00
• 5f557a7f9f test: add tests for the new feature Karim shoair 2026-02-07 01:28:58 +02:00
• 66476d42be feat(browsers): Add option to block requests to specific domains Karim shoair 2026-02-07 01:26:21 +02:00
• d00c34ee7f style(parser): Improve the type hint for find_by_text and find_by_regex Karim shoair 2026-02-06 03:19:00 +02:00
• 05f62fba81 fix(response): Force the body of the response to always be bytes Karim shoair 2026-02-06 03:14:23 +02:00
• 9acc8194c5 test: update the tests accordingly Karim shoair 2026-02-06 02:45:13 +02:00
• aa7a95fb70 feat(parser)!: Make all selection return selector objects by default Karim shoair 2026-02-06 02:43:51 +02:00
• d8fe909c8b docs: make translation urls absolute Karim shoair 2026-02-05 02:44:12 +02:00
• 151e4e7a7c docs: adding new sponsor Karim shoair 2026-02-05 02:39:51 +02:00
• f88502718f feat(proxy control): Force a proxy at request level at any given point Karim shoair 2026-02-02 14:26:52 +02:00
• 6e0277f23b test: add new tests for the proxy rotation logic Karim shoair 2026-02-02 00:16:40 +02:00
• ef8c5bc7d6 feat(spiders/fetchers): Adding proxy rotation logic and change retry logic Karim shoair 2026-02-02 00:16:22 +02:00
• 65f1a23358 fix(browsers): Improve stealth and speed by adjusting flags Karim shoair 2026-01-29 00:58:04 +02:00
• 0c3d359b46 fix(browsers): disable autoplay for all browsers Karim shoair 2026-01-29 00:57:30 +02:00
• 90c52c45c7 feat(parser): replacing tldextract with tld library Karim shoair 2026-01-23 00:00:48 +02:00
• 5dbbd84b3d fix(requests): return the cached fingerprint on second call Karim shoair 2026-01-20 20:28:13 +02:00
• d90189e675 test: update tests accordingly Karim shoair 2026-01-20 20:22:47 +02:00
• 122923c80e feat(spiders/requests): Use a reliable method for fingerprint generation Karim shoair 2026-01-20 20:21:42 +02:00
• 5aa178a76e style: update post requests type hints to be more accurate Karim shoair 2026-01-20 18:27:52 +02:00
• d0f52c3855 test: remove the tests for the removed html_utils module Karim shoair 2026-01-20 16:37:17 +02:00
• 95955af3be build: Add w3lib to deps Karim shoair 2026-01-20 16:36:34 +02:00
• 0ec78e2335 test: update tests accordingly Karim shoair 2026-01-19 16:40:18 +02:00
• 6947066761 feat(spiders): Make graceful shutdown always enabled Karim shoair 2026-01-19 16:40:02 +02:00
• 94bc759c6a test: updates tests to include new logic Karim shoair 2026-01-19 16:13:29 +02:00
• 25850db92c fix(spiders): handle errors with manual pause during stream mode Karim shoair 2026-01-19 16:13:03 +02:00
• fbc8be5bcc fix(spiders): fix a bug with session stats counter Karim shoair 2026-01-19 16:00:06 +02:00
• 4ecf8a64d3 fix(spiders): Fix for when the pausing system has large queue Karim shoair 2026-01-19 15:04:40 +02:00
• cf0de7ea6d Merge branch 'main' into v4 Karim shoair 2026-01-19 14:47:59 +02:00
• 9229cef895 docs: adding new sponsor Karim shoair 2026-01-19 14:12:32 +02:00
• d247b5b691 test: add tests for the spiders system Karim shoair 2026-01-19 03:46:00 +02:00
• 0d003f62e3 ops: update bandit checklist Karim shoair 2026-01-19 03:45:39 +02:00
• 1f0f86e3a2 feat(spiders): Add pause/resume system for crawls Karim shoair 2026-01-19 03:43:10 +02:00
• acdb39988c ops: update bandit checklist Karim shoair 2026-01-19 03:40:57 +02:00
• bf5aa021d4 fix(browsers): solving an issues with leaving playwright loop open when cdp connection fails Karim shoair 2026-01-18 16:46:48 +02:00
• 02c9dffc77 style(spiders): make parse function abstract Karim shoair 2026-01-18 00:01:04 +02:00
• afe6b2e97b feat(spiders): Change items hook to make it for processing items + add a stat for this Karim shoair 2026-01-17 20:52:49 +02:00
• 41c7a7e793 fix(spiders engine): Improve logging Karim shoair 2026-01-17 19:57:04 +02:00
• dac854a129 fix(parser): handle responses with empty body Karim shoair 2026-01-13 02:35:26 +02:00
• 35d570120a feat(spiders): Add option to logging format and date format Karim shoair 2026-01-12 17:11:52 +02:00
• 3466bcab2c feat: add hook to run after collecting an item Karim shoair 2026-01-12 15:58:33 +02:00
• e0f94cf7d0 feat(spiders): Add a stream mode to collect items in realtime Karim shoair 2026-01-12 15:53:29 +02:00
• 927cfd85ff feat(spiders): The option to start sessions lazily Karim shoair 2026-01-12 02:09:50 +02:00
• f2c6216968 style: remove unwanted import from spiders main init file Karim shoair 2026-01-12 01:21:26 +02:00
• 72c3c2792a feat(spiders): add simple export system for the results Karim shoair 2026-01-12 01:21:05 +02:00
• a17bb1d976 feat(browsers): Add option to retry tabs that gives errors Karim shoair 2026-01-11 23:40:26 +02:00
• 5a11c2466b fix: make retry_blocked_request take the response as well Karim shoair 2026-01-11 22:54:17 +02:00
• 736873a7c6 feat(spiders): Add logic to detect blocked requests and retry them Karim shoair 2026-01-11 21:56:57 +02:00
• 774364b6d3 style(spiders): make the usage of uvloop optional Karim shoair 2026-01-11 19:57:03 +02:00
• 0bfdab7158 feat: spiders use uvloop/winloop when available Karim shoair 2026-01-11 18:39:42 +02:00
• d5b9ed11b2 fix(browsers): Close pages that give error Karim shoair 2026-01-11 18:34:13 +02:00
• 32f4f2adf4 build: Add AnyIO to the dependencies Karim shoair 2026-01-11 17:01:33 +02:00
• 059a708b6d feat(spiders system): a prototype of the new spiders system Karim shoair 2026-01-11 16:53:18 +02:00
• dd5c30c521 feat(response): Option to set the referer as the previous response url. Karim shoair 2026-01-11 03:24:30 +02:00
• 1443a469a3 fix: handle a bug with response follow referer flow argument Karim shoair 2026-01-11 03:22:51 +02:00
• aa4817f349 feat: Change logger to be flexible enough for the spiders classes Karim shoair 2026-01-11 03:22:20 +02:00
• c5bc82deb8 fix(browsers): solve a bug with setting referer on the request level Karim shoair 2026-01-11 03:21:03 +02:00
• 8e36962177 ops: add local Claude files to gitignore Karim shoair 2026-01-10 20:41:23 +02:00
• 0634f5796c feat(spiders): Add follow function to the response Karim shoair 2026-01-10 20:39:43 +02:00
• f80e8828f7 refactor: internal API changes Karim shoair 2026-01-08 01:53:48 +02:00
• 3a360acced build: pump version up temporary Karim shoair 2026-01-08 01:37:47 +02:00
• 08f3e6efad fix: add missing line Karim shoair 2026-01-08 01:01:00 +02:00
• 6553d90e0f ops: update workflows to run for this branch Karim shoair 2026-01-08 00:53:29 +02:00
• 9eb5f68fcb test: update tests accordingly Karim shoair 2026-01-08 00:52:35 +02:00
• 6a73f9dcd7 refactor: internal API changes to be easily used as indicators for spiders Karim shoair 2026-01-08 00:52:06 +02:00
• de79fe80bb v0.3.14 Karim shoair 2026-01-03 22:29:20 +02:00
• 2f87507afe test: update database test to suite windows Karim shoair 2026-01-03 22:02:03 +02:00
• f99baf65f7 test: remove the check for incognito Karim shoair 2026-01-03 01:50:46 +02:00
• c8a2456b81 fix(StealthyFetcher): disable incognito mode to solve #123 Karim shoair 2026-01-03 01:50:02 +02:00
• 30a77de2be build: pump version up and browserforge version Karim shoair 2026-01-03 01:48:34 +02:00
• 6ad2ecb9f2 v0.3.13 Karim shoair 2026-01-01 22:07:00 +02:00
• b5fdb71b52 docs: improving the API reference section Karim shoair 2026-01-01 21:07:20 +02:00
• 4947152be2 docs: change docstring style for some methods Karim shoair 2026-01-01 21:06:35 +02:00
• 9742b239be docs: fix for rtd build Karim shoair 2026-01-01 17:42:47 +02:00
• e275509ece docs: fix for rtd build Karim shoair 2026-01-01 17:05:10 +02:00
• 185a9c594f docs: fix for rtd build Karim shoair 2026-01-01 17:01:24 +02:00
• c6e4482837 docs: Improve docs website Karim shoair 2026-01-01 16:56:57 +02:00
• cb5051ca56 docs: update contributing page Karim shoair 2026-01-01 03:18:15 +02:00
• 4d478eb84c docs: update the donate page Karim shoair 2026-01-01 03:16:10 +02:00
• 2d5e2f41c4 docs: update dev articles Karim shoair 2026-01-01 03:14:12 +02:00
• 0a3bb6cfdb docs: update the migrating from BS article Karim shoair 2025-12-31 18:32:34 +02:00
• 55ff20c238 docs: update replacing AI article Karim shoair 2025-12-31 18:28:28 +02:00
• 7d5b907d6c docs: adding more data to the adaptive feature page Karim shoair 2025-12-31 16:56:57 +02:00
• 17202d23a8 docs: Update the MCP server page Karim shoair 2025-12-31 16:32:21 +02:00
• ad234165e9 docs: update the cli section Karim shoair 2025-12-31 03:00:26 +02:00
• 3b9a6d021e fix: correct cli argument Karim shoair 2025-12-31 03:00:06 +02:00
• d4ae6c49c0 docs: update the StealthyFetcher page Karim shoair 2025-12-31 00:22:33 +02:00
• f6f0ed4b37 fix(async stealth): wait the coroutine of the bounding box Karim shoair 2025-12-31 00:19:53 +02:00
• 33c5633c58 test: correcting tests with the new arguments Karim shoair 2025-12-30 22:06:37 +02:00
• 0c7e26ae5c fix(fingerprints): do not create google referer for IPs Karim shoair 2025-12-30 21:58:09 +02:00
• a2b5628a73 docs: update DynamicFetcher page Karim shoair 2025-12-29 22:02:01 +02:00
• 67e8f4d515 docs: update DynamicFetcher page Karim shoair 2025-12-29 20:48:45 +02:00
• e1cac0f69f docs: update requests page Karim shoair 2025-12-29 20:48:32 +02:00
• bc420da19b docs: update deps Karim shoair 2025-12-29 20:39:13 +02:00
• ee13af9516 build: update deps Karim shoair 2025-12-29 16:08:51 +02:00
• 6c25dccb05 docs: updating the Fetcher Basics and the HTTP requests pages Karim shoair 2025-12-28 21:57:21 +02:00