[Web API type definition issue] VideoEncoderConfig interface should have hevc attribute

[yaruno]

Summary

VideoEncoderConfig interface is missing hevc attribute

Expected vs. Actual Behavior

Modern browsers widely support hevc codec for video encoding and decoding https://developer.mozilla.org/en-US/docs/Web/Media/Guides/Formats/Video_codecs#hevc_h.265. Supported on Chrome 107+, Edge 18+, Firefox 120+, Opera 94+, Safari 11+

The lack of hevc attribute for video encoder means that users cannot change the format of encoded video to e.g. annex-b which is favoured for realtime video streaming.

VideoEncoder interface looks now following:

interface VideoEncoderConfig {
    alpha?: AlphaOption;
    avc?: AvcEncoderConfig;
    bitrate?: number;
    bitrateMode?: VideoEncoderBitrateMode;
    codec: string;
    contentHint?: string;
    displayHeight?: number;
    displayWidth?: number;
    framerate?: number;
    hardwareAcceleration?: HardwareAcceleration;
    height: number;
    latencyMode?: LatencyMode;
    scalabilityMode?: string;
    width: number;
}

Should be:

interface VideoEncoderConfig {
    alpha?: AlphaOption;
    avc?: AvcEncoderConfig;
    bitrate?: number;
    bitrateMode?: VideoEncoderBitrateMode;
    codec: string;
    contentHint?: string;
    displayHeight?: number;
    displayWidth?: number;
    framerate?: number;
    hardwareAcceleration?: HardwareAcceleration;
    height: number;
    hevc?: HevcEncoderConfig;
    latencyMode?: LatencyMode;
    scalabilityMode?: string;
    width: number;
}

interface HevcEncoderConfig {
    format?: HevcBitstreamFormat;
}

type HevcBitstreamFormat = "annexb" | "hevc";

Removing following from removedTypes.jsonc and fixing tests should alleviate this:

"VideoEncoderConfig": {
    "members": {
        "member": {
            "hevc": null // Blink only as of 2023-03
        }
    }
}

Playground Link

No response

Browser Support

• This API is supported in at least two major browser engines (not two Chromium-based browsers).

Have Tried The Latest Releases

• This issue applies to the latest release of TypeScript.
• This issue applies to the latest release of @types/web.

Additional Context

No response

[saschanaz]

That MDN page does not show that Gecko and WebKit supports HEVC video encoding. Actually none of them supports it. (They have only avc.

• Gecko: https://searchfox.org/mozilla-central/rev/5813af297bb0e6581ab49029ba897c3e12a9134c/dom/webidl/VideoEncoder.webidl#77
• WebKit: https://searchfox.org/wubkat/rev/98733fd3822325456ee066f400c6465b5fc8dd91/Source/WebCore/Modules/webcodecs/WebCodecsVideoEncoderConfig.idl#47

[yaruno]

Thank you for posting the Gecko and WebKit repos here. You are right that in these repos there's no mention of HEVC support. However, it's interesting that Safari does mention that it supports webcodecs HEVC since 17.4 https://developer.apple.com/documentation/safari-release-notes/safari-17_4-release-notes .

If you have a safari browser 17.4+ you can test out following on console to verify encoder support for hevc main, Level 4.1 codec.

(async () => { if (!window.VideoEncoder) { console.warn('WebCodecs VideoEncoder not exposed – Safari < 17.4'); return; } const hevcMain = { codec: 'hvc1.1.6.L93.B0', width: 1920, height: 1080, bitrate: 8_000_000, framerate: 30 }; const support = await VideoEncoder.isConfigSupported(hevcMain); console.log(support); })();

Tested on safari 18.4.

I wonder where the discrepancy is from, as this domain is bit out of my scope.

[HolgerJeromin]

Webkit is the engine.
With that Apple builds Safari.
This is the same as every chrome-based Browser can have different codecs included.
Often they are using the codecs from the host OS (for performance and / or licencing issues).

[saschanaz]

Perhaps they do support HEVC but not the HEVC specific configuration?

[yaruno]

Perhaps they do support HEVC but not the HEVC specific configuration?

That is possible, but it is interesting if that's the case. The difference between annex-b and non-annex b format is basically adding binary separators before Network Abstraction Layer (NAL) units instead of communicating before a video stream or during the video stream separately where the relevant information for decoding is available, which is computationally and programmatically quite straightforward to do. I'd like to imagine that the apple hardware already has support for annexb format video encoding, but I wonder why would it not be reflected on the webkit repo.

[yaruno]

Webkit is the engine. With that Apple builds Safari. This is the same as every chrome-based Browser can have different codecs included. Often they are using the codecs from the host OS (for performance and / or licencing issues).

Thank you for the clarification.

[yaruno]

I'll need to have a look at https://github.com/WebKit/WebKit/blob/main/Source/WebCore/Modules/webcodecs/WebCodecsVideoEncoder.cpp , maybe hevc annex-b support could be added to it and then I can revisit this issue on later date.