fix(adapters): 识别 200 状态的挑战 / 人机验证页,解析失败前兜底,失败页面留样
- detectChallenge 支持页面类请求(GET + Accept text/html)的 2xx 响应:正文只先读 256 KB, 短页且没有种子表 / 用户信息块时,认 Cloudflare 挑战标题 / 脚本与 Turnstile / hCaptcha / reCAPTCHA 门页; 带登录表单的页面不算门页。非页面请求不读正文 - siteFetch 用上面的判定;挑战错误附脱敏页面样本,文案带级联结果(x-umt-challenge) - spider 适配器:认不出列表 / 详情时先看是不是人机验证页,是就报 challenge(「搜索页需要人机验证」), 不再报改版;parse_error / challenge 都经 deps.onFailureSample 交出脱敏样本(前 64 KB,去掉个人信息块、 passkey / authkey / uid / 用户名 / Cookie / 邮箱) - classifyChallengePage 也认门页(浏览器过挑战时不再把门页当已通过) Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Cxwgnn2pbzQ7rXv3rq7fei
This commit is contained in:
1 parent
4fa001750b
commit
1779e1a0f2
11 files changed
+2078
-34
No files matched your search
@@ -1,10 +1,17 @@
|
||||
export {
|
||||
CHALLENGE_OUTCOME_HEADER,
|
||||
CHALLENGE_PEEK_BYTES,
|
||||
CHALLENGE_STATUSES,
|
||||
type ChallengePageKind,
|
||||
challengeGateRule,
|
||||
challengeOutcomeText,
|
||||
challengeRuleOfHeaders,
|
||||
challengeRuleOfHtml,
|
||||
challengeRuleOfOkPage,
|
||||
challengeRuleOfUnrecognizedPage,
|
||||
classifyChallengePage,
|
||||
detectChallenge,
|
||||
isPageRequest,
|
||||
} from "./challenge.ts";
|
||||
export {
|
||||
audiencesDefinition,
|
||||
@@ -34,6 +41,13 @@ export {
|
||||
structureDescription,
|
||||
} from "./detail-description.ts";
|
||||
export { type FormattedDetailPage, formatDetailPage } from "./detail-format.ts";
|
||||
export {
|
||||
buildFailureSample,
|
||||
FAILURE_SAMPLE_MAX_BYTES,
|
||||
redactPageHtml,
|
||||
type SiteFailureSample,
|
||||
type SiteRequestKind,
|
||||
} from "./failure-sample.ts";
|
||||
export {
|
||||
type FeedMediaType,
|
||||
normalizeSiteFeedItem,
|
||||
|
||||
Reference in new issue
Block a user