fix(adapters): 识别 200 状态的挑战 / 人机验证页,解析失败前兜底,失败页面留样

- detectChallenge 支持页面类请求(GET + Accept text/html)的 2xx 响应:正文只先读 256 KB,
  短页且没有种子表 / 用户信息块时,认 Cloudflare 挑战标题 / 脚本与 Turnstile / hCaptcha / reCAPTCHA 门页;
  带登录表单的页面不算门页。非页面请求不读正文
- siteFetch 用上面的判定;挑战错误附脱敏页面样本,文案带级联结果(x-umt-challenge)
- spider 适配器:认不出列表 / 详情时先看是不是人机验证页,是就报 challenge(「搜索页需要人机验证」),
  不再报改版;parse_error / challenge 都经 deps.onFailureSample 交出脱敏样本(前 64 KB,去掉个人信息块、
  passkey / authkey / uid / 用户名 / Cookie / 邮箱)
- classifyChallengePage 也认门页(浏览器过挑战时不再把门页当已通过)

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Cxwgnn2pbzQ7rXv3rq7fei
This commit is contained in:
rootandClaude Opus 5.5 committed 2026-10-06 15:35:31 +08:00
1 parent 4fa001750b
commit 1779e1a0f2
11 files changed
+2078 -34

No files matched your search

+14
View File
@@ -1,10 +1,17 @@
export {
CHALLENGE_OUTCOME_HEADER,
CHALLENGE_PEEK_BYTES,
CHALLENGE_STATUSES,
type ChallengePageKind,
challengeGateRule,
challengeOutcomeText,
challengeRuleOfHeaders,
challengeRuleOfHtml,
challengeRuleOfOkPage,
challengeRuleOfUnrecognizedPage,
classifyChallengePage,
detectChallenge,
isPageRequest,
} from "./challenge.ts";
export {
audiencesDefinition,
@@ -34,6 +41,13 @@ export {
structureDescription,
} from "./detail-description.ts";
export { type FormattedDetailPage, formatDetailPage } from "./detail-format.ts";
export {
buildFailureSample,
FAILURE_SAMPLE_MAX_BYTES,
redactPageHtml,
type SiteFailureSample,
type SiteRequestKind,
} from "./failure-sample.ts";
export {
type FeedMediaType,
normalizeSiteFeedItem,