修复多源热榜抓取稳定性并规范忽略本地补丁目录#138
Conversation
There was a problem hiding this comment.
Code Review
This pull request significantly hardens the data fetching logic for several routes, including Baidu, Coolapk, Huxiu, Kuaishou, and Weibo, by implementing multi-layered fallback mechanisms and switching to more resilient endpoints or HTML parsing. Key improvements include the integration of TopHub as a secondary data source and the addition of detailed change records for better maintainability. Review feedback highlights opportunities to improve ID stability in the Coolapk fallback, ensure consistent error messaging in English, validate mandatory fields during Kuaishou parsing, and refine exception handling to ensure all fallback layers are correctly executed.
| return; | ||
| } | ||
| data.push({ | ||
| id: href.split("/").pop() || title, |
| @@ -74,15 +83,12 @@ const getList = async (noCache: boolean) => { | |||
| : "未知错误"; | |||
| throw new Error(`快手数据解析失败: ${msg}`); | |||
| allItems.forEach((item: { id: string }) => { | ||
| // 基础数据 | ||
| const hotItem = jsonObject[item.id]; | ||
| if (!hotItem) return; |
| const apolloData = parseApolloList(result.data || ""); | ||
| const htmlData = apolloData.length ? apolloData : parseRenderedRankList(result.data || ""); |
There was a problem hiding this comment.
当前的“分层降级”逻辑存在一个小缺陷:如果 parseApolloList 抛出异常(例如页面完全没有 APOLLO 状态),执行流会直接跳到外层的 catch 块并进入 TopHub 降级,从而跳过了 parseRenderedRankList(DOM 解析)这一层。建议局部捕获 Apollo 解析的异常,以便能按预期尝试 DOM 解析。
let apolloData: ListItem[] = [];
try {
apolloData = parseApolloList(result.data || "");
} catch (e) {
// 忽略 Apollo 解析错误,尝试后续的 DOM 解析
}
const htmlData = apolloData.length ? apolloData : parseRenderedRankList(result.data || "");
变更说明
本次修改将此前仅覆盖
dist/routes/*.js的本地补丁,正式迁回源码层的 TypeScript 路由实现,避免重新构建后补丁失效,并提升多个热榜源在接口变动或风控场景下的可用性。主要改动
baidu路由:增强s-data解析兼容性,并补充更稳的链接兜底逻辑huxiu路由:由脆弱 API 切换为页面 HTML 解析kuaishou路由:保留 APOLLO 主链路,并增加 DOM fallback 与 TopHub fallbackweibo路由:切换为移动端接口,并在失败时回退到 TopHubcoolapk路由:保留官方 API,并在失败时回退到 TopHub.gitignore:忽略.planning/、.pnpm-store/、补丁/等本地工作目录产物验证结果
已在本地完成以下验证:
pnpm lintpnpm buildbaidu/huxiu/kuaishou/weibo/coolapk备注
weibo、coolapk在当前验证环境中主要通过 fallback 链路返回数据,说明降级逻辑已生效