import { createWriteStream } from 'node:fs'; import { rename } from 'node:fs/promises'; import { join } from 'node:path'; import { Readable } from 'node:stream'; import { pipeline } from 'node:stream/promises'; import { createGunzip } from 'node:zlib'; import { IMDB_DATASETS_BASE, IMDB_RATINGS, ROOT } from '../config.js'; const FILE = 'title.ratings.tsv'; export async function syncImdbRatings() { const url = `${IMDB_DATASETS_BASE}/${FILE}.gz`; const tmpPath = join(ROOT, `${FILE}.tmp`); console.log(`Downloading: "${url}"`); const res = await fetch(url); if (!res.ok || !res.body) { throw new Error(`Failed to fetch ${url}: HTTP ${res.status}`); } await pipeline(Readable.fromWeb(res.body), createGunzip(), createWriteStream(tmpPath)); await rename(tmpPath, IMDB_RATINGS); console.log(`Wrote ${IMDB_RATINGS}`); } if (import.meta.url === `file://${process.argv[1]}`) { syncImdbRatings().catch((err) => { console.error(err); process.exit(1); }); }