The image encoding layer of HEIF is the same HEVC, but the container is MPEG-4 Part 12 (the Quicktime-descendant ISO Base Media Format; the core behind .mp4, .3gp, etc.), which theoretically gives it wide support. In this structure, HEIF supports image sequences, which will help it compete with animated GIF, GIFV (which is really just a short MPEG-4 Part 14 file containing usually an H.264 video track), animated WebP (who uses these?), and WebM.
This format doesn't really blaze new ground (EDIT: it does in the sense that it defines a container format to express image-y constructs like still images and sequences of images in an ISO media container, but see my other comment that asks how this is similar but different to video [3]), but if this repackaging and the resulting code donation and political clout helps it gain traction, we still would gain a lot.
The problem, of course, is always with backwards-compatibility. WebP was aggressively promoted by Google the same way Microsoft used to promote its quirky Windows Media formats back in the early 2000s, but the WebP and non-WebP camps are still largely separate. This is unfortunate, because WebP in my opinion really isn't very good, and a backwards-compatible compressor on top of JPEG, such as Dropbox's Lepton [2] achieves similar results. Any HEVC-based image compressor is necessarily incompatible with JPEG; so broad buy-in would be required from makers of software and hardware products, from operating systems, file managers, image viewers, image editors, cameras, and the like, to really enjoy its improved compression rates, vs. being just another incompatible format that only functions inside controlled ecosystems.
The video codec AV1 currently in development was supposed to be a grand alliance of disparate companies to agree on a common format for the future, and rectify the WebM vs. non-WebM split, but continuing to promulgate sophisticated formats based on work outside of this scope (such as MPEG- or ITU-derived custom formats) just muddies this further.
[1] https://bellard.org/bpg/ [2] https://news.ycombinator.com/item?id=13230805 [3] https://news.ycombinator.com/item?id=14490734
Apple is just blatantly not interested in cooperating with the rest of the market. Mozilla, Google and Microsoft are all part of AoM, along with a bunch of hardware companies and distributors yet Apple is absent and apparently pursuing HEVC instead.
This isn't new either, Apple pushed HLS when others were pushing DASH and MPEG-TS when others were pushing fragmented MP4.
They seem really determined to go their own way and ignore everyone else for some reason.
And many more companies support it over AoM: http://www.mpegla.com/main/programs/HEVC/Pages/Licensors.asp... http://www.mpegla.com/main/programs/HEVC/Pages/Licensees.asp... http://www.hevcadvance.com/pdfnew/LicensorList.pdf http://www.hevcadvance.com/pdfnew/LicenseeList.pdf
Including the likes of: BBC, Samsung, Dolby, GE, MediaTek, Philips, Mitsubishi, Warner Bros, Sony etc. And as for hardware vendors well AMD, Intel, Nvidia etc all support H.265.
There is just less expectations to be able to interoperate via "dumb files" in the "app ecosystem" where the apps run in their own isolated sandboxes and import/export are explicit actions.
Not saying that this is the best thing, but the situation is different enough from the time when WebP/WMP were introduced that HEIF actually might have a fighting chance. The fact that HEIF appears to be far bigger improvement over JPEG than previous efforts of course also helps.
You need to think about HEIF as the container and disconnected from the codec used to compress the media content. In that sense, HEIF is a massive trailblazer. It gives us an ISO standard to describe image and mixed media consistent with existing methods (i.e. MP4) while allowing for new constructs (i.e. useful features of the future in addition to tiling, depth map, stereo, etc.) and new codecs as they are developed and adopted. HEIF is a universal forward-looking format.
Until now, almost all of the major imaging formats were tied to the codec and not terribly extensible to new use cases. While BPG is a great format in the short term (it gave us HEVC coded images around 2.5 years ago), it isn't an ideal choice for the long term when viewed as above.
BTW: Did you know that you can embed LPCM/μ-Law PCM/ADPCM audio data in JPEG/Exif?
In the past, I articulated, on the topic of 'Ask HN: Software for writing a diary that will be around in 20 years from now' [2]:
I'd be wary of the archival potential of formats that are solely used on the Web with little usage on tangible physical hardware by major commercial publishers -- the Web of today moves very fast and technologies come and go. Google is pushing WebP, WebM, but work is already under way on a big consensus format called AV1. When AV1 comes out, new VP8/VP9 content will likely no longer be produced. Browsers periodically prune older features, 20 years is almost as long as the web has been around, and given enough time support for the format may only be available in software that make format coverage an explicit goal (ffmpeg, libav, VLC). Opus is being made a mandatory audio codec for WebRTC, teleconferencing is usually ephemeral -- will there be lots of .opus files sitting on disks in the future? Too early to tell, not worth gambling on.
The context of this was choosing formats for the express purpose of long-term archival, but our incidental usage of formats today will shape the sort of files that will be naturally around in our future.
[1] https://news.ycombinator.com/item?id=5589206 [2] https://news.ycombinator.com/item?id=12979854#12980359
If we can get everyone to agree on a format that handles layers, exposure bracketing, transparency, animation, high bit depth, different color models, both lossy and lossless compression, every common type of metadata, etc., that would be great.
What's really different between this and the existing usages of the MP4 container for images, though? Is it just the packaging as an "image" format, or specification of more image-y metadata?
You are right -- the main difference is that BMFF was originally designed for time-based media (video, audio) but isn't in itself well suited towards media that isn't time based, like images and their associated metadata (e.g. Exif).
HEIF extends the ISO BMFF so that you can have untimed media -- one or more photos, a collection -- or even mixed media comprising both untimed and timed media. Apple's live photo is a good example of mixed media, comprising a photo and a video.
But the HEIF container format goes so much further. You could have image sequences -- for example a higher-quality version of Animated GIF (using I-frames only, or P/B for more compression) with looping -- tiling, auxiliary images like depth maps and alpha channels, or even stereo images with a left and right channel.
Though originally HEIF came out of the HEVC standards track, hence the words "High Efficiency" in the name, it was later extended to include other codecs like JPEG, AVC. There's no reason it couldn't be extended to include VP9, PNG, or any other codec.
Think of HEIF as a versatile, extensible, standardized container format. The media coding scheme is separable. This is a big deal for the future of image/media coding because we're no-longer locked into "yet another format" that's tied to the codec. With the ISO BMFF box model, HEIF can grow to adapt new constructs and codecs for some generations to come.
Image formats are already some of the biggest attack surface of modern systems, "executable" image formats (hello PDF) have an absolutely ghastly track record there.
And not being able to open an image if you don't have an internet connection sounds dreadful.
Re: requiring an Internet connection—you still do require an Internet connection to display an image, if it's in a format whose decoder you don't currently have installed; you just currently have to manually 1. figure out what the format even is, 2. select a library package for a decoder for that format, and 3. install that package.
Presumably such libraries would be able to register decoder-URLs they are (non-reference) implementations of—like they register MIME types today—so your system would be able to display standard formats no problem; it'd just be the weird rare or new ones that would trigger the zero-install process for a decoder.
Caching the decoder smells like having already downloaded the image library. Mostly just different in that "first run time" is now always the same thing as "install time".
Wikipedia also does this to enable video playback on Edge and Safari [2].
The idea to do this within the format itself is quite old. The very early versions of what became Vorbis had a very similar concept, with programmable passes. Matroska once had a field for a link to download the VfW decoder dll (the horror!). It's never really caught on, due to reasons already listed by sibling comments.
[1] https://arewecompressedyet.com/analyzer/?maxFrames=4&decoder...
The HEIF examples on the Nokia HEIF site do something similar; they implement a HEVC decoder (and do the HEIF container processing) all in JavaScript using http://www.libde265.org. The images you see are decoded into a HTML5 canvas.
What's so bad about WebP? It's not the best thing in the universe but it beats JPEG and PNG and has a wide support base. With a shim it has native support in most browsers anyways since WebP is a just a single frame of video. And as a bonus you don't have to worry about someone suing you for royalties down the road.
WebP is based on VP8, which is now obsolete. It was meant to be a H.264 competitor, a generation behind H.265 and VP9/VP10. WebP compression efficiency is much closer to JPEG than H.265.
This format beats WebP by a margin wider than WebP was smaller than JPEG.
* Lack of clear spec ("The code is the spec")
* Lack of versioning of spec (at least 3 major versions with different capabilities/features of which not all degrade nicely)
* Forced 4:2:0 chroma subsampling
* 8bit-channel only (no support for 10bit+ aka wide-gamut images)
[1] https://github.com/nokiatech/heif/blob/master/LICENSE.TXT
And either way VP9 and AV1 are simply inferior codecs to H.265 in almost every way. And as for royalty free well that's because MPEG-LA has't ever sued but they did give a royalty free license to Google for patents infringed by VP9. And since Google accepted it does indicate that AV1 is likely to also be patent encumbered.
We would all like a royalty free codec but it isn't happening anytime soon. And so failing that I would prefer to go with the codec with the best support which by far is H.265.
Source?
AV1 is already beating HEVC in compression and it's not even finalized yet: https://youtu.be/lzPaldsmJbk?t=2416
> "If you [...] order or agree to the institution of patent litigation [...] against any entity [...] alleging that any of these implementations of WebM or any code incorporated within any of these implementations of WebM constitutes direct or contributory patent infringement, [...] then any patent rights granted to you under this License for these implementations of WebM shall terminate as of the date such litigation is filed."
This is an (significant) improvement over JPEG.
Why is that? Every edit you make to that photo means that the photo will degrade in quality. I always thought JPEG artifacts and compression lossage was a bad thing for photos, not a good thing?
"FLIF does not have a lossy mode, but interlaced files can be decoded progressively so we can simply use something like dd if=lossless.flif of=lossy.flif bs=1024 count=5"
You can see this in the examples where BPG does well in stream loading: https://nokiatech.github.io/heif/technical.html
HEIF would probably be similar to BPG in performance.
Still FLIF looks pretty darn interesting.
I don't know what you mean by well. As far as I'm concerned lossy FLIF beats JPEG easily. Here is a comparision:
https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding#P...
If you choose to implement this format, I hope you enjoy negotiating licenses with at least four different entities so that you can display pictures.
Worse, there is a financial incentive to implement only a subset of the decoders - HEVC is more expensive than AVC which is more expensive than JPEG. Even things like 4:4:4 color are more expensive than 4:2:0 color for HEVC.
It's a standard format, it's faster than the competition, it has just as good (or better) quality vs compression metrics...
... all it needs is support and it'll take off. Apple is a good start for that.
Compared to this, HEIF is a different way of packaging HEVC frames; namely into ISOBMFF (MPEG-4 Part 12), the most basic level of MPEG container. This comes with benefits [2] that come along with using that particular container, but also drawbacks, because with some of the advanced features of HEIF you're essentially describing a sequence of frames -- which is almost like video -- but you're doing it in a way incompatible with the way you'd describe video. I'd be curious if the two "representations" are losslessly convertible, for example.
[1] https://bellard.org/bpg/ [2] https://nokiatech.github.io/heif/technical.html
They are: you could just extract the HEVC NAL units, and re-write them into a MP4 or QuickTime container, making sure to properly place the codec configuration box, etc.
HEIF also goes beyond a sequence of frames, in that it can describe alpha planes, depth maps, tiling, etc. In that case there might not be an analog with a standard video. If you really wanted to decompose a HEIF container, you might choose to extract the raw media into elementary streams (for HEVC or AVC; or if you're using the JPEG codec, just plain JPEG files) adjacent to any metadata like Exif, etc. This is essentially what Nokia's conformance files are [1].
[1] https://github.com/nokiatech/heif_conformance/tree/master/bi...
It's more like HEIF is for HEVC what WebP is for AV1.
However, the HEIF wrapper for H.265 strikes me as quite complex. It brings baggage of ISO wrapper formats and tries to support everything ever crammed into any image-ish format. That, in addition to patent licensing, may be another difficulty in widespread support for the format. It will be hard to implement robust, secure and fully-featured players.
Besides the coding efficiency, JPEG really hit the big time because of the readily available libjpeg cross-platform implementation. In this case, HEIF can leverage existing implementations of the ISO Base Media File Format box model; it builds heavily on that standard.
Nokia is doing the right thing by releasing their format handling library, though perhaps they could loosen up their license to include commercial use. ;-) Hopefully there are some developers amongst us inspired enough to start a new open project that, like libjpeg, brings HEIF to the masses.
Could have been branded better if I'm honest. I know engineers generally don't care about stuff like this but some of us will have to say this term many times a day if takes off.
