fill out the Option module?

[jmagaram]

I like using Option. I'm wondering if the module should be filled out with some functions that are available in Option packages from other languages. Most of these functions are simple conveniences that aren't strictly necessary. But they lead to shorter and sometimes easier-to-understand code. If we provide conversion functions from both sides, like option->null and null->option, the programmer finds it more easily. Maybe standardize on terminology for lazy forms. These usually take just a few lines to implement. What do you think? Should we consider augmenting the Option module? Similar changes could be made to Result as well. I've looked at F# and fp-ts. Here are some ideas. If there is interest in this, I can do more research and make a proposal.

module type Ideas = {
  let valueExn: option<'a> => 'a // same as getExn
  let valueUnsafe: option<'a> => 'a
  let valueOrDefault: (option<'a>, 'a) => 'a
  let valueOrDefaultWith: (option<'a>, unit => 'a) => 'a // new lazy form
  let concat: (option<'a>, option<'a>, ('a, 'a) => 'a) => option<'a>
  let orElseWith: (option<'a>, unit => option<'a>) => option<'a> // lazy form
  let map2: (option<'a>, option<'b>, ('a, 'b) => 'c) => option<'c>
  let map3: (option<'a>, option<'b>, option<'c>, ('a, 'b, 'c) => 'd) => option<'d>
  let fold: (option<'a>, 'b, ('a, 'b) => 'b) => 'b
  let match: (option<'a>, 'b, 'a => 'b) => 'b // same as mapWithDefault?
  let matchWith: (option<'a>, unit => 'b, 'a => 'b) => 'b
  let iter: (option<'a>, 'a => unit) => unit
  let exists: (option<'a>, 'a) => bool
  let contains: (option<'a>, 'a) => bool
  let count: option<'a> => int
  let flatten: option<option<'a>> => option<'a>
  let toArray: option<'a> => array<'a>
  let toList: option<'a> => list<'a>
  let toUndefined: option<'a> => Js.undefined<'a>
  let toNull: option<'a> => Js.null<'a>
  let fromUndefined: Js.undefined<'a> => option<'a>
  let fromNull: Js.null<'a> => option<'a>
  let fromPredicate: ('a, 'a => bool) => option<'a>
}

module Option = {
  let fromPredicate = (a, pred) => a->Some->Option.filter(pred)

  let concat = (a, b, f) => {
    switch (a, b) {
    | (a, None) => a
    | (None, b) => b
    | (Some(a), Some(b)) => Some(f(a, b))
    }
  }

  let fold = (i, j, f) => {
    switch i {
    | None => j
    | Some(i) => f(i, j)
    }
  }
}

module Examples = {
  let nums = [Some(5), None, Some(9)]

  let sum1 = nums->Array.reduce(0, (sum, next) => next->Option.fold(sum, (i, j) => i + j))
  let sum2 = nums->Array.reduce(None, (sum, next) => sum->Option.concat(next, (i, j) => i + j))

  let s2 = (s: string) =>
    s
    ->String.trim
    ->String.concat("abc")
    ->String.replace("a", "x")
    ->Option.fromPredicate(i => String.length(i) <= 50)
}

[jmagaram]

I surveyed F#, Rust, Belt, fp-ts, and OCaml. I also looked at the Javascript Array functions because an Option is sometimes like an array with either zero or one item in it. Here is what I found.

https://1drv.ms/x/s!AjaZ-t3uLVN_0PAoyJH_Lj_zZBmu9w

I think this encompasses the universe of potential functions we'd want to add to the Option module. Certainly don't want to add them all. I didn't understand a lot of the fp-ts type classes and so maybe good stuff is going to be ignored, but only for the hard-core functional programmers. I'll post my initial suggestions next.

[jmagaram]

Here are some thoughts on a complete package and some usage examples. A few notes:

• Most other packages include a fold/reduce
• I don't understand how the cmp and eq methods that we've got now are supposed to be used. They seem awkward for instantiating a Belt Map or Set or sorting an array
• I like having functions that treat option like an array of 0 or 1. F#, Rust does it. So things like exists, contains, length are handy. And reduce/fold too.
• A variety of import/export functions are useful, like toArray, fromResult. If we have things, should they be on both sides for discovery, like Result.toOption? Rust and fp-ts have a lot of these convenient conversions.
• Lazy forms are helpful, like getting the value inside or a lazy get of a default.
• map2 seems especially useful. F# even has a map3. This is kind of like a concat.
• Rust has interesting functions that treat option like boolean -> xor, and. We've got orElse. Not sure these others are practically useful.
• Transpose functions, like array of option to option of array, are kind of interesting.
• If we do U uncurried, should they be under Option.U.filter or Option.filterU. Would keep things cleaner in a separate sub-module.
• Many of these are not strictly necessary. For example, exists can be built using mapOr. But I think the intent is clearer and code sometimes shorter with the helper functions.

Maybe we should collaborate on the doc I posted? I can make it editable. Let me know how to proceed or general thoughts.

module type Options = {
  type t<'a> = option<'a>

  let cmp: (t<'a>, t<'a>, ('a, 'a) => int) => int
  let cmpBy: (('a, 'a) => int, t<'a>, t<'a>) => int // new
  let eq: (t<'a>, t<'a>, ('a, 'a) => bool) => bool
  let eqBy: (('a, 'a) => bool, t<'a>, t<'a>) => bool // new

  let contains: (t<'a>, 'a) => bool // array.includes
  let exists: (t<'a>, 'a => bool) => bool // array.some
  let isSome: t<'a> => t<'a>
  let isNone: t<'a> => t<'a>
  let isNoneOr: (t<'a>, 'a => bool) => bool // array.every

  let expect: (t<'a>, ~message: string) => t<'a> // new

  let forEach: (t<'a>, 'a => unit) => unit
  let inspect: (t<'a>, 'a => unit) => t<'a> // new

  let getExn: t<'a> => 'a
  let getOr: (t<'a>, 'a) => 'a
  let getOrWith: (t<'a>, unit => 'a) => 'a // new
  let getUnsafe: t<'a> => 'a

  let length: t<'a> => int // new

  let filter: (t<'a>, 'a => bool) => t<'a>
  let map: (t<'a>, 'a => 'b) => t<'b>
  let flatMap: (t<'a>, 'a => t<'b>) => t<'b>
  let map2: (t<'a>, t<'b>, ('a, 'b) => 'c) => t<'c> // new
  let mapOr: (t<'a>, 'b, 'a => 'b) => 'b
  let mapOrWith: (t<'a>, unit => 'b, 'a => 'b) => 'b // new
  let orElse: (t<'a>, t<'a>) => t<'a>
  let orElseWith: (t<'a>, unit => t<'a>) => t<'a> // new
  let andAlso: (t<'a>, t<'a>) => t<'a> // new
  let xor: (t<'a>, t<'a>) => t<'a> // new
  let flat: t<'a> => t<'a> // new

  let reduce: (t<'a>, ('b, 'a) => 'b, 'b) => 'b // new
  let reduceBack: ('b, ('a, 'b) => 'b, t<'a>) => 'b // new
  let transposeArray: array<t<'a>> => t<array<'a>> // new
  let transposeList: list<t<'a>> => t<list<'a>> // new
  let transposeResult: t<result<'ok, 'err>> => result<t<'ok>, 'err> // new

  let fromNull: null<'a> => t<'a> // new
  let fromPredicate: ('a => bool, 'a) => t<'a> // new
  let fromResult: result<'ok, 'err> => t<'ok> // new
  let fromError: result<'ok, 'err> => t<'err> // new
  let fromUndefined: undefined<'a> => t<'a> // new

  let toArray: t<'a> => array<'a> // new
  let toList: t<'a> => list<'a> // new
  let toNull: t<'a> => null<'a> // new
  let toUndefined: t<'a> => undefined<'a> // new
  let toResult: (t<'ok>, 'err) => result<'ok, 'err> // new
  let toResultWith: (t<'ok>, unit => 'err) => result<'ok, 'err> // new
}

module Examples = (O: Options) => {
  module Exists = {
    let isFirstNameValid1 = O.exists(_, i => String.length(i) < 20)
    let isFirstNameValid2 = O.mapOr(_, false, i => String.length(i) < 20)
  }

  module IsNoneOr = {
    let isMiddleNameValid1 = O.isNoneOr(_, i => String.length(i) < 20)
    let isMiddleNameValid2 = O.mapOr(_, true, i => String.length(i) < 20)
  }

  module Inspect = {
    let q1 =
      Some(54)->O.inspect(i => Console.log(`Saw the value ${i->Int.toString}`))->O.map(i => i * 2)
    let q2 =
      Some(54)
      ->O.map(i => {
        Console.log(`Saw the value ${i->Int.toString}`)
        i
      })
      ->O.map(i => i * 2)
  }

  module FromPredicate = {
    let validate1 = O.fromPredicate(i => String.length(i) < 20)
    let validate2 = i => String.length(i) < 20 ? Some(i) : None
    let validate3 = i => i->Some->O.filter(i => String.length(i) < 20)
    let r1 = "mike"->O.fromPredicate(i => String.length(i) < 20, _)
    let r2 = "mike"->(i => String.length(i) < 20 ? Some(i) : None)
  }

  module Length = {
    let x1 = Some(3)->O.length
    let x2 = Some(3)->O.mapOr(0, _ => 1)
  }

  module Map2 = {
    let add1 = (x, y) => O.map2(x, y, (i, j) => i + j)
    let add2 = (x, y) =>
      switch (x, y) {
      | (Some(i), Some(j)) => Some(i * j)
      | (_, _) => None
      }
  }

  module Reduce = {
    let maybeNum = Some(54)
    let num = 23

    let x1 = maybeNum->O.reduce((i, j) => i + j, num)
    let x2 = maybeNum->O.mapOr(num, i => i + num)

    let y1 = num->O.reduceBack((i, j) => i + j, maybeNum)
    let y2 = num->(_ => maybeNum->O.mapOr(num, i => i + num))
  }

  module CmpBy = {
    [Some(1), None]->Array.sortInPlace(O.cmpBy((i, j) => i < j ? -1 : i > j ? 1 : 0))
    [Some(1), None]->Array.sortInPlace((i, j) => O.cmp(i, j, (i, j) => i < j ? -1 : i > j ? 1 : 0))
  }
}

[glennsl]

Great research! I think this could be very useful, but it's a bit hard to work with in its current form. Would it be possible to make it into a matrix, with the different functions listed in one dimension, and languages in the other? Something like this:

rescript-core	JavaScript	OCaml	F#	Rust
flat	flat	-	flatten	flattten
flatMap	-	bind	bind	and_then
forEach	forEach	iter	iter	iter

[jmagaram]

Ok I'll do that. I'll try a standardized name for each and make a pivot table. Hadn’t looked at rust before. Very cool! Interesting and weird ideas there for option like transpose and xor and Boolean and. Fp-ts has a very interesting one fromTryCatch where you give it a function and it wraps it in try catch and puts the value in Some if it succeeds and None if it throws.

…

________________________________ From: Glenn Slotte ***@***.***> Sent: Wednesday, March 8, 2023 7:28:01 AM To: rescript-association/rescript-core ***@***.***> Cc: jmagaram ***@***.***>; Author ***@***.***> Subject: Re: [rescript-association/rescript-core] fill out the Option module? (Issue #85) Great research! I think this could be very useful, but it's a bit hard to work with in its current form. Would it be possible to make it into a matrix, with the different functions listed in one dimension, and languages in the other. Something like this: rescript-core JavaScript OCaml F# Rust flat flat - flatten flattten flatMap - bind bind and_then forEach forEach iter iter iter — Reply to this email directly, view it on GitHub<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Frescript-association%2Frescript-core%2Fissues%2F85%23issuecomment-1460333272&data=05%7C01%7C%7C963f6b7713e94756b07d08db1fe9b4b1%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638138860835289195%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=KJT%2FEn0UCIyWmDAIgkIo3FmBQBhE6QUbtabvZ3Ej2Y4%3D&reserved=0>, or unsubscribe<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FADLZYS5NZF5RF7WVBQ6VGMDW3CQQDANCNFSM6AAAAAAVPF2ZGU&data=05%7C01%7C%7C963f6b7713e94756b07d08db1fe9b4b1%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638138860835289195%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=z%2FvhmCBIkl39KR7hUJi45HMHVNoXZ%2FerupLTmJHURVs%3D&reserved=0>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[jmagaram]

I updated the spreadsheet if you want to play around with it. But here is the data. It's not totally accurate - I got lost in some of the fp-ts type classes and Rust has traits and there is the undefined/null confusion, but this is pretty close. I think next step is to collaboratively work on a resi file so we can see/build the whole picture. I don't know what tool is good for that. What I did was scan the spreadsheet, pluck out the things that looked useful into a resi, try to give them some consistent names and then write some sample code that used them to see if the ergonomics were good. That's what you see earlier in this issue thread. I'd probably take out the and and xor and add in a fromTryCatch. Tweak it until it seems like a complete and consistent proposal.

Function	Belt	Core	F#	fp-ts	Array	oCaml	Rust
cmp	1	1	1	1		1	1
contains			1		1		1
eq	1	1	1	1		1
filter	1	1	1	1	1		1
flat			1	1	1	1	1
flatMap	1	1	1	1	1	1	1
forEach	1	1	1		1	1	1
forEachInspect							1
fromError				1
fromNull			1	1
fromOk				1
fromPredicate				1
fromResult				1
fromTryCatch				1
isNone	1	1	1	1		1	1
isNoneOr			1		1
isSome	1	1	1	1		1	1
isSomeAnd			1	1	1		1
length			1		1
logicalAnd							1
logicalOr	1	1	1				1
logicalOrLazy			1				1
logicalXor							1
map	1	1	1	1	1	1	1
map2			1		1		1
map3			1
mapOr	1	1					1
mapOrLazy							1
reduce			1	1	1	1
reduceBack			1	1	1
toArray			1
toIterable						1
toList			1			1
toNull			1	1
toResult						1	1
toResultLazy							1
toUndefined				1
transposeArray				1
transposeResult							1
unzip							1
valueExn	1	1	1			1	1
valueExnMessage							1
valueOr	1	1	1			1	1
valueOrLazy			1	1			1
valueOrNaturalDefault							1
valueUnsafe	1	1					1
zip							1
(blank)			2	32	5	2	13

[jmagaram]

We could do a VS Code Live Share. Haven't played with it much and get on our phones to talk. Or a Design doc folder in the repo where we can put things like my spreadsheet, proposals, etc. I have an idea what a good .resi would be for the Option module but this doesn't seem like it should be a pull request for the actual code since some higher level discussion/thinking needs to happen around it.

[zth]

Thank you for mapping that out. I'll keep this open for now, but please note that we're not looking to expand the surface of Core right now unless it's clear that it's for fixing bugs, or inconsistencies between things already in Core.

[jmagaram]

Here is my completed proposal. When the time comes to expand the API surface, let me know and I'm happy to write some code and tests and docs for any/all of these. I will need your feedback. At the moment the API surface for option is very incomplete/weak and doesn't satisfy the goal "rich enough (without being bloated) so that you don't need to reach for anything else for typical ReScript development." I've had to write my own "OptionUtil" to fill the gaps. We could prioritize a bunch of this. For example, low priority would be all the transpose functions, fromTryCatch, some of the conversions - I'm not sure what versions of toNull/Undefined/Nullable are really important for interoperability, and maybe some of the lazy forms. High priority is exists, isNoneOr/forAll, map2, concat, flat, and fold/reduce. Most of it is trivial to code and test.

module type Options = {
  type t<'a> = option<'a>

  let cmp: (t<'a>, t<'a>, ('a, 'a) => int) => int
  let eq: (t<'a>, t<'a>, ('a, 'a) => bool) => bool

  let contains: (t<'a>, 'a) => bool // like array.includes
  let exists: (t<'a>, 'a => bool) => bool // new
  let isSome: t<'a> => t<'a>
  let isNone: t<'a> => t<'a>
  let isNoneOr: (t<'a>, 'a => bool) => bool // like array.every
  let length: t<'a> => int // like array.length

  let forEach: (t<'a>, 'a => unit) => unit

  let getExn: t<'a> => 'a
  let getExnMsg: (t<'a>, string) => 'a // rust.expect
  let getOr: (t<'a>, 'a) => 'a
  let getOrWith: (t<'a>, unit => 'a) => 'a // new
  let getUnsafe: t<'a> => 'a

  let filter: (t<'a>, 'a => bool) => t<'a>
  let map: (t<'a>, 'a => 'b) => t<'b>
  let flatMap: (t<'a>, 'a => t<'b>) => t<'b>
  let map2: (t<'a>, t<'b>, ('a, 'b) => 'c) => t<'c> // new
  let map3: (t<'a>, t<'b>, t<'c>, ('a, 'b, 'c) => 'd) => t<'d> // new
  let mapOr: (t<'a>, 'b, 'a => 'b) => 'b
  let mapOrWith: (t<'a>, unit => 'b, 'a => 'b) => 'b // new
  let concat: (t<'a>, t<'a>, ('a, 'a) => 'a) => t<'a> // new
  let flat: t<t<'a>> => t<'a> // new
  let xor: (t<'a>, t<'a>) => t<'a> // new
  let or: (t<'a>, t<'a>) => t<'a>
  let orWith: (t<'a>, unit => t<'a>) => t<'a> // new

  let fold: (t<'a>, ('b, 'a) => 'b, 'b) => 'b // new
  let foldBack: ('b, ('a, 'b) => 'b, t<'a>) => 'b // new

  let transposeArray: array<t<'a>> => t<array<'a>> // new
  let transposeArrayWith: (array<'a>, 'a => t<'b>) => t<array<'b>>
  let transposeList: list<t<'a>> => t<list<'a>> // new
  let transposeListWith: (list<'a>, 'a => t<'b>) => t<list<'b>> // new
  let transposeResult: t<result<'ok, 'err>> => result<t<'ok>, 'err> // new

  let fromPredicate: ('a => bool, 'a) => t<'a> // new
  let fromResult: result<'ok, 'err> => t<'ok> // new
  let fromError: result<'ok, 'err> => t<'err> // new
  let fromTryCatch: (unit => 'a) => t<'a> // new
  let fromNull: null<'a> => t<'a> // new
  let fromUndefined: undefined<'a> => t<'a> // new
  let fromNullable: nullable<'a> => t<'a> // new

  let toArray: t<'a> => array<'a> // new
  let toList: t<'a> => list<'a> // new
  let toNull: t<'a> => null<'a> // new
  let toUndefined: t<'a> => undefined<'a> // new
  let toResult: (t<'ok>, 'err) => result<'ok, 'err> // new
  let toResultWith: (t<'ok>, unit => 'err) => result<'ok, 'err> // new
}