Skip to content

Commit 7cbcfe4

Browse files
author
Thomas Hurst
committed
Handle HTTP chunking across multi-byte boundaries
When a response contains multi-byte UTF-8 characters that cross chunk boundaries, appending the two halves to the intermediate string results in two Unicode Replacement Characters being inserted instead of the two halves being joined. Use StringDecoder to buffer the partial characters across chunks. Resolves #22
1 parent 1cdcf24 commit 7cbcfe4

File tree

1 file changed

+4
-1
lines changed

1 file changed

+4
-1
lines changed

src/utils.ts

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
import { version } from "../version.ts";
22
import https from "node:https";
33
import qs from "node:querystring";
4+
import { StringDecoder } from "node:string_decoder";
45
import { RequestTimeoutError } from "./errors.ts";
56

67
/**
@@ -63,15 +64,17 @@ export function execute(
6364
return new Promise((resolve, reject) => {
6465
let timer: number;
6566
const req = https.get(url, (resp) => {
67+
const decoder = new StringDecoder("utf8");
6668
let data = "";
6769

6870
// A chunk of data has been recieved.
6971
resp.on("data", (chunk) => {
70-
data += chunk;
72+
data += decoder.write(chunk);
7173
});
7274

7375
// The whole response has been received. Print out the result.
7476
resp.on("end", () => {
77+
data += decoder.end();
7578
try {
7679
if (resp.statusCode == 200) {
7780
resolve(data);

0 commit comments

Comments
 (0)