[FEATURE] Support union type for basic types

[badmonster0]

We want to support union type for basic types.

Background

Union types are supported in Python (e.g. str | int). CocoIndex doesn't support it now, so when a library already defines a dataclass with Union type members, users cannot use the existing type as argument / return type of a CocoIndex function (existing example from relik).

Alternative solution considered

Bind union type in Python to Json type in CocoIndex

However, Json type is too generic and loses essential type information. Unexpected values can be set to the field without being noticed earlier (e.g. when calling LLM to create the dataclass value).

With this said, binding an unsupported type to Json still can be supported, because there're still types that cannot be supported even after we support union type for basic types. This will be treated as a separate feature request.

Not limit union type to basic type

i.e. we may support union type for struct/table types too.

We may support it eventually, but it's difficult to reliably interpret the type for a specific struct value, especially our struct types are unnamed (for now).

Things to Change

We want to add Union as a new variant for BasicValueType, e.g.

Union(Vec<BasicValueType>),

We considered putting it as a new type at the same level as BasicValueType, StructType and TableType. One practical reason of not doing so is: things like list[str | int] will not be representable. Considering Vector is already part of BasicValueType, having Union as another BasicValueType makes sense.

Other code to update:

• Methods related BasicValueType, for serializing, deserializing, etc.
• Value representation doesn't need to change, but when we decode values we're taking target type as argument in BasicValue::from_json(). It needs to updated.
• JsonSchemaBuilder
• Python SDK code to update: typing.py, maybe also convert.py.
  Please also the test test_convert.py
• Update the document data_types.mdx

---

❤️ Contributors, please refer to 📙Contributing Guide.
Unless the PR can be sent immediately (e.g. just a few lines of code), we recommend you to leave a comment on the issue like I'm working on it or Can I work on this issue? to avoid duplicating work. Our Discord server is always open and friendly.

[chardoncs]

Hello, I'm working on it.

I have done some experiment in my fork, and it seems doable.