Skip to content

feat: add utils/parse-ndjson #1394

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
a7e750c
#1075 added feat: parse-ndjson
Feb 26, 2024
7c3c7f0
chore: update copyright years
stdlib-bot Feb 27, 2024
5286d31
Update README.md
maniksharma17 Feb 27, 2024
4b3d5c3
Update benchmark.js
maniksharma17 Feb 27, 2024
679e74f
Update README.md
maniksharma17 Feb 28, 2024
3e5e726
Update main.js
maniksharma17 Feb 28, 2024
188e413
Update index.d.ts
maniksharma17 Feb 28, 2024
e5517cb
Merge branch 'stdlib-js:develop' into feat/1075-added-parse-ndjson
maniksharma17 Feb 28, 2024
4bed10f
Update repl.txt
maniksharma17 Feb 28, 2024
034f275
Update benchmark.js
maniksharma17 Feb 28, 2024
2a3c709
Update repl.txt
maniksharma17 Feb 28, 2024
5adc6c3
Update repl.txt
maniksharma17 Feb 28, 2024
181155c
feat: corrected indentation, header and corrected function declaration
shubhexists Feb 28, 2024
7d0946b
feat: corrected tests, example and repl
shubhexists Feb 28, 2024
9e5c544
Update README.md
maniksharma17 Feb 28, 2024
8eb3c9f
Update README.md
maniksharma17 Feb 28, 2024
4f82b02
Update README.md
maniksharma17 Feb 29, 2024
334595c
README.md updated
maniksharma17 Feb 29, 2024
a101d9a
README.md updated
maniksharma17 Feb 29, 2024
765842f
README.md updated
maniksharma17 Feb 29, 2024
dd79535
README.md updated
maniksharma17 Feb 29, 2024
5969f4c
README.md updated
maniksharma17 Feb 29, 2024
edcd8d5
README.md updated
maniksharma17 Feb 29, 2024
8ae3540
README.md updated
maniksharma17 Feb 29, 2024
96e56e5
chore: ci error package.json fixed
shubhexists Feb 29, 2024
94fb9c1
chore: fixed license
shubhexists Feb 29, 2024
313f51a
Update lib/node_modules/@stdlib/utils/parse-ndjson/package.json
maniksharma17 Mar 2, 2024
5cb8248
Update lib/node_modules/@stdlib/utils/parse-ndjson/README.md
maniksharma17 Mar 2, 2024
3a29ca5
Update lib/node_modules/@stdlib/utils/parse-ndjson/benchmark/benchmar…
maniksharma17 Mar 2, 2024
ff0a2df
Update lib/node_modules/@stdlib/utils/parse-ndjson/docs/types/index.d.ts
maniksharma17 Mar 2, 2024
6c16d36
Update lib/node_modules/@stdlib/utils/parse-ndjson/docs/types/index.d.ts
maniksharma17 Mar 2, 2024
f6d752c
Update lib/node_modules/@stdlib/utils/parse-ndjson/lib/main.js
maniksharma17 Mar 2, 2024
fb7428e
Update lib/node_modules/@stdlib/utils/parse-ndjson/lib/main.js
maniksharma17 Mar 2, 2024
6e0664e
Update main.js
maniksharma17 Mar 2, 2024
5533e9f
Update lib/node_modules/@stdlib/utils/parse-ndjson/docs/repl.txt
maniksharma17 Mar 2, 2024
99e172d
Update lib/node_modules/@stdlib/utils/parse-ndjson/docs/repl.txt
maniksharma17 Mar 2, 2024
8d4db86
Update lib/node_modules/@stdlib/utils/parse-ndjson/docs/repl.txt
maniksharma17 Mar 2, 2024
80853bd
Update lib/node_modules/@stdlib/utils/parse-ndjson/docs/types/index.d.ts
maniksharma17 Mar 2, 2024
afd31a4
Update lib/node_modules/@stdlib/utils/parse-ndjson/docs/types/index.d.ts
maniksharma17 Mar 2, 2024
40f91a4
Update lib/node_modules/@stdlib/utils/parse-ndjson/lib/index.js
maniksharma17 Mar 2, 2024
e85bffd
Update lib/node_modules/@stdlib/utils/parse-ndjson/examples/index.js
maniksharma17 Mar 2, 2024
1409bcd
Update lib/node_modules/@stdlib/utils/parse-ndjson/examples/index.js
maniksharma17 Mar 2, 2024
0e2a8ad
Update lib/node_modules/@stdlib/utils/parse-ndjson/examples/index.js
maniksharma17 Mar 2, 2024
7264919
Update lib/node_modules/@stdlib/utils/parse-ndjson/examples/index.js
maniksharma17 Mar 2, 2024
826bde9
Update index.js
maniksharma17 Mar 2, 2024
92d2f2e
Update lib/node_modules/@stdlib/utils/parse-ndjson/docs/types/index.d.ts
maniksharma17 Mar 2, 2024
90472cc
Update lib/node_modules/@stdlib/utils/parse-ndjson/lib/main.js
maniksharma17 Mar 2, 2024
8375edc
Update lib/node_modules/@stdlib/utils/parse-ndjson/test/test.js
maniksharma17 Mar 2, 2024
9bb2cf5
Update lib/node_modules/@stdlib/utils/parse-ndjson/test/test.js
maniksharma17 Mar 2, 2024
8e3eb1f
Added support for trailing newline and alternate characters
maniksharma17 Mar 6, 2024
f6f7416
Resolved lint errors
maniksharma17 Mar 6, 2024
ee945ee
Resolved lint errors
maniksharma17 Mar 6, 2024
b7d8c6a
Resolved lint errors
maniksharma17 Mar 6, 2024
9b780d5
Resolved lint errors
maniksharma17 Mar 6, 2024
76c6be0
Update README.md
maniksharma17 Mar 7, 2024
8f01e53
Update index.js
maniksharma17 Mar 7, 2024
2064f36
Update main.js
maniksharma17 Mar 8, 2024
1f76d20
Update test.js
maniksharma17 Mar 8, 2024
8af4a02
Apply suggestions from code review
Planeshifter Mar 8, 2024
eb2b06b
Update lib/node_modules/@stdlib/utils/parse-ndjson/docs/types/test.ts
Planeshifter Mar 8, 2024
f128dba
Update lib/node_modules/@stdlib/utils/parse-ndjson/lib/main.js
Planeshifter Mar 8, 2024
3efc890
Update lib/node_modules/@stdlib/utils/parse-ndjson/lib/main.js
Planeshifter Mar 8, 2024
21e58d9
Update lib/node_modules/@stdlib/utils/parse-ndjson/docs/repl.txt
Planeshifter Mar 8, 2024
c112f7f
Update lib/node_modules/@stdlib/utils/parse-ndjson/examples/index.js
Planeshifter Mar 8, 2024
1fc6a35
Update index.js
Planeshifter Mar 8, 2024
ea33c68
Update test.js
Planeshifter Mar 8, 2024
00af9d3
Update benchmark.js
maniksharma17 Mar 8, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
176 changes: 176 additions & 0 deletions lib/node_modules/@stdlib/utils/parse-ndjson/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,176 @@
<!--

@license Apache-2.0

Copyright (c) 2024 The Stdlib Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

-->

# parseNDJSON

> Parse a string containing serialized newline-delimited [JSON][json] (NDJSON).

<section class="usage">

## Usage

```javascript
var parseNDJSON = require( '@stdlib/utils/parse-ndjson' );
```

#### parseNDJSON( str\[, reviver] )

Parses a `string` as `newline-delimited JSON`.

```javascript
var out = parseNDJSON( '{"beep":"boop"}\n{"example":42}' );
// returns [ { 'beep': 'boop' }, { 'example': 42 } ]
```

To transform the `string` being parsed, provide a `reviver`.

```javascript
function reviver( key, value ) {
if ( key === '' || key === 'beep' ) {
return ( typeof value === 'string' ) ? value.toUpperCase() : value;
}
return ( typeof value === 'number' ) ? value * 2 : value;
}

var str = '{"beep":"boop"}\n{"value": 20}\n{"numbers": [1,2,3]}';
var out = parseNDJSON( str, reviver );
// returns [ { 'beep' : 'BOOP' }, { 'value': 40 }, { 'numbers': [ 2, 4, 6 ] } ]
```

</section>

<!-- /.usage -->

<section class="notes">

## Notes

- In contrast to the native [`JSON.parse()`][json-parse], this implementation parses `string` as `newline-delimited JSON` and returns an array of parsed JSONs.

```javascript
var out = JSON.parse( '{"beep":"boop"}\n{"foo":"baz"}' );
// throws <SyntaxError>

out = parseNDJSON( '{"beep":"boop"}\n{"foo":"baz"}' );
// returns [ { 'beep': 'boop' }, { 'foo': 'baz' } ]
```


- In contrast to the native [`JSON.parse()`][json-parse], this implementation throws a TypeError if provided any value which is not a `string`.

```javascript
var out = JSON.parse( null );
// returns null

out = parseNDJSON( null );
// throws <TypeError>
```


- In contrast to the native [`JSON.parse()`][json-parse], this implementation does **not** throw a SyntaxError if unable to parse a string as newline-delimited JSON.

```javascript
var out = parseNDJSON( '{"beep":boop}' );
// returns <SyntaxError>

out = JSON.parse( '{"beep":boop}' );
// throws <SyntaxError>
```


- In contrast to the native [`JSON.parse()`][json-parse], this implementation throws a TypeError if provided a reviver argument which is not a function.

```javascript
var out = JSON.parse( '{"a":"b"}', [] );
// returns { 'a': 'b' }

out = parseNDJSON( '{"a":"b"}', [] );
// throws <TypeError>
```


</section>

<!-- /.notes -->

<section class="examples">

## Examples

<!-- eslint no-undef: "error" -->

```javascript
var parseNDJSON = require( '@stdlib/utils/parse-ndjson' );

var out = parseNDJSON( '{"name":"John"}\n{"name":"Doe"}' );
// returns [ { 'name': 'John' }, { 'name': 'Doe' } ]

function reviver( key, value ) {
if ( key === 'name' ) {
return value.toUpperCase();
}
return value;
}

out = parseNDJSON( '{"name":"John"}\n{"name":"Doe"}', reviver );
// returns [ { 'name': 'JOHN' }, { 'name': 'DOE' } ]

out = parseNDJSON( '{"name":John}\n{"name":Doe}' );
// returns <SyntaxError>

out = parseNDJSON( ' ' );
// returns []

out = parseNDJSON( '{}' );
// returns [ {} ]

out = parseNDJSON( '{"name":"Eve"}\n42\ntrue\n[1,2,3]' );
// returns [ { 'name': 'Eve' }, 42, true, [ 1, 2, 3 ] ]

out = parseNDJSON( '{"name":"John"}\r\n{"name":"Doe"}' );
// returns [ { 'name': 'John' }, { 'name': 'Doe' } ]

out = parseNDJSON( '{"name":"John"}\n{"name":"Doe"}\n' );
// returns [ { 'name': 'John' }, { 'name': 'Doe' } ]
```

</section>

<!-- /.examples -->

<!-- Section for related `stdlib` packages. Do not manually edit this section, as it is automatically populated. -->

<section class="related">

</section>

<!-- /.related -->

<!-- Section for all links. Make sure to keep an empty line after the `section` element and another before the `/section` close. -->

<section class="links">

[json]: http://www.json.org/

[json-parse]: https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Global_Objects/JSON/parse

</section>

<!-- /.links -->
61 changes: 61 additions & 0 deletions lib/node_modules/@stdlib/utils/parse-ndjson/benchmark/benchmark.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
/**
* @license Apache-2.0
*
* Copyright (c) 2024 The Stdlib Authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

'use strict';

// MODULES //

var bench = require( '@stdlib/bench' );
var fromCodePoint = require( '@stdlib/string/from-code-point' );
var pkg = require( './../package.json' ).name;
var parseNDJSON = require( './../lib' );


// MAIN //

bench( pkg, function benchmark( b ) {
var str;
var out;
var i;
var j;

b.tic();

for ( i = 0; i < b.iterations; i++ ) {
// Generate an NDJSON string with a changing property key in each line:
str = '{"beep":"boop","'+fromCodePoint( 97 + (i%26) ) + '":true}\n{"example":' + i + '}';
out = parseNDJSON( str );

if ( out !== out ) {
b.fail( 'should return an array of JSON objects' );
}
}

b.toc();

for ( j = 0; j < out.length; j++ ) {
if ( out[j] instanceof Error ) {
b.fail( 'should return an array of JSON objects' );
}
}

b.pass( 'benchmark finished' );
b.end();
});

// TODO: Add benchmarks with different sized NDJSON strings
45 changes: 45 additions & 0 deletions lib/node_modules/@stdlib/utils/parse-ndjson/docs/repl.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@

{{alias}}( str[, reviver] )
Attempts to parse a string as newline-delimited JSON (NDJSON).

Function behavior differs from `JSON.parse()` as follows:

- Returns array of parsed JSON
- Throws a `TypeError` if provided any value which is not a string.
- Returns a `SyntaxError` if unable to parse a string as NDJSON.
- Throws a `TypeError` if provided a `reviver` argument which
is not a function.

Parameters
----------
str: string
String to parse as newline-delimited JSON.

reviver: Function (optional)
Transformation function.

Returns
-------
out: Array | Error
Array of parsed values or an error.

Examples
--------
> var obj = '{"beep":"boop"}\n{"example":42}\n{"data":[1,2,3]}';
> var result = {{alias}}( obj )
[ { 'beep': 'boop' }, { 'example': 42 }, { 'data': [ 1, 2, 3 ] } ]

// Provide a reviver:
> function reviver( key, value ) {
... if ( key === '' || key === 'beep' ) {
... return ( typeof value === 'string' )
... ? value.toUpperCase() : value;
... };
... return typeof value === 'number' ? value * 2 : value;
... };
> var ndjsonString = '{"beep":"boop"}\n{"example":42}\n{"data":[1,2,3]}';
> var resultWithReviver = {{alias}}( ndjsonString, reviver )
[ { 'beep': 'boop' }, { 'example': 84 }, { 'data': [ 2, 4, 6 ] } ]

See Also
--------
38 changes: 38 additions & 0 deletions lib/node_modules/@stdlib/utils/parse-ndjson/docs/types/index.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
/*
* @license Apache-2.0
*
* Copyright (c) 2024 The Stdlib Authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

// TypeScript Version: 4.1

/**
* Parses a string as newline-delimited JSON (NDJSON).
*
* @param str - input string containing NDJSON
* @param reviver - transformation function applied to each line
* @returns array of parsed values or an error
*
* @example
* var arr = parseNDJSON( '{"beep":"boop"}\\n{"example":42}\\n{"data":[1,2,3]}' );
* // returns [ { 'beep': 'boop' }, { 'example': 42 }, { 'data': [ 1, 2, 3 ] } ]
*/

declare function parseNDJSON( str: string, reviver?: Function ): Array<any> | Error;


// EXPORTS //

export = parseNDJSON;
54 changes: 54 additions & 0 deletions lib/node_modules/@stdlib/utils/parse-ndjson/docs/types/test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
/**
* @license Apache-2.0
*
* Copyright (c) 2024 The Stdlib Authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

import parseNDJSON = require( './index' );


// TESTS //

// The function returns an array of parsed values...
{
const validNDJSON = '{"beep":"boop"}\n{"example":42}\n{"data":[1,2,3]}';
parseNDJSON( validNDJSON ); // $ExpectType any[] | Error
}

// The function does not compile if the argument is a value other than a string...
{
parseNDJSON( true ); // $ExpectError
parseNDJSON( false ); // $ExpectError
parseNDJSON( 5 ); // $ExpectError
parseNDJSON( [] ); // $ExpectError
parseNDJSON( {} ); // $ExpectError
parseNDJSON( ( x: number ): number => x ); // $ExpectError
}

// The function does not compile if the second argument is a value other than a function...
{
parseNDJSON( '{"beep":"boop"}\n{"example":42}', true ); // $ExpectError
parseNDJSON( '{"beep":"boop"}\n{"example":42}', false ); // $ExpectError
parseNDJSON( '{"beep":"boop"}\n{"example":42}', 5 ); // $ExpectError
parseNDJSON( '{"beep":"boop"}\n{"example":42}', [] ); // $ExpectError
parseNDJSON( '{"beep":"boop"}\n{"example":42}', {} ); // $ExpectError
parseNDJSON( '{"beep":"boop"}\n{"example":42}', 'baz' ); // $ExpectError
}

// The compiler throws an error if the function is provided an unsupported number of arguments...
{
parseNDJSON( ); // $ExpectError
parseNDJSON( '{"beep":"boop"}', 'baz', 'foo' ); // $ExpectError
}
Loading