Show HN: Detect upsampling in images (opens in new tab)

(github.com)

113 points0x0910y ago15 comments

15 comments

15 comments · 9 top-level

vortico10y ago· 2 in thread

This is fantastic, and I was just wondering if something like existed a week ago. Sometimes I need to find the original picture of a thumbnail on the internet, so I use Google reverse image search to search for other sizes of my uploaded image. However, some websites simply upscale the low quality image in order to be at the top of the search results. I wonder if Google can use a technology like this to place those results on a lower ranking.

vonklaus10y ago

Interesting, is that how google rev img search works?

I was trying to find a usecase for this tech. I thought google maybe hashed files and searched thatway, but thumbnails dont always turn up results. I thought maybe they just used pagesource data or metadata to find the same pictures because it doesnt work on all images. Maybe this library can fix that

vortico10y ago

It's more complicated than a hash, but who knows what their proprietary engine uses. Here are some guesses. https://www.quora.com/What-is-the-algorithm-used-by-Googles-...

Regardless of how that process works, once Google scrapes an image, they can store some metadata similar to the output of your software, so that it may be used for sorting the much smaller set of images returned by the image similarity algorithm. The top result can then be deemed the "most original" image since it contains the most detail, and the webpage it was found on can get a boost of pagerank or something.

joshu10y ago· 2 in thread

Could you do this over a sliding window and calculate a score for regions? That would let you see if something has been composited from up scaled images.

0x09OP10y ago

That's exactly how it works. The example doesn't show more than one, but it will return multiple resolutions ranked with a confidence percent (the meaning of which depends on the method) as long as it's above a reasonable threshold.

Usually multiple results are a misdetection -- for example heavily compressed JPEGs will mess it up pretty badly:

    $ convert Lenna.png -quality 50 Lenna.jpg
    $ resdet Lenna.jpg
    given: 512x512
    best guess: 448x384
    all width        height
      448 (77.49%)     384 (90.19%)
      384 (66.99%)     448 (89.78%)
      320 (60.68%)     320 (72.54%)
      388 (55.92%)     256 (56.43%)
      445 (55.49%)   
      345 (55.47%)   
      242 (55.06%)

(the block-based nature of JPEG compression/quantization creates a similar zeroing effect along every 1/8 of the spectrum)

joshu10y ago

Sure. But I meant for each region, show the underlying resolution of that area. So like for FFT of a whole sample vs FFT of short periods to make a spectrogram.

mozumder10y ago· 1 in thread

You can detect downsampling by looking at chroma frequencies. Bayer pattern color filters on cameras mean that color resolution is never going to be as high as pixel resolution (assuming 1:1 mapping of pixels to sensor pixels) and if you see chroma frequencies that are higher than 2x pixel densities, then you have some downsampling going on.

mey10y ago

Not all cameras use the Bayer pattern (there are alternatives) and Canon (and I believe Nikon) are looking at ditching it entirely

http://petapixel.com/2015/10/29/canon-120mp-sensor-reportedl...

sigmar10y ago· 1 in thread

Does anyone know of something similar for automatic analysis of audio using the FFT? There are visual ways to do it, but it would be nice to have an automated script

0x09OP10y ago

Late reply but you can actually do this with the library as is. It's not designed for it so you'll have to use fftw yourself and call some borderline internal stuff, but it ought to work the same.

    float* samples = ... //load audio samples here
    int len = ...
    fftwf_plan p = fftwf_plan_r2r_1d(len,samples,samples,FFTW_REDFT10,FFTW_ESTIMATE);
    fftwf_execute(p);

    RDResolution* res;
    size_t count;
    RDMethod* m = resdet_methods();
    m->func(samples,len,1,1,&res,&count,12,m->threshold);

stevetjoa10y ago

For anyone interested in this area, the IEEE Transactions on Information Forensics and Security (TIFS) [1] is full of papers on the so-called area of image forensics -- the identification of processing that has been performed upon an image, including resampling, rotation, JPEG compression, block processing, and more.

Consequently, another hot research area is image anti-forensics -- the obfuscation of such operations to avoid detection. (Shameless self-promotion: one example paper on anti-forensics of JPEG compression can be found here [2].)

[1] http://www.signalprocessingsociety.org/publications/periodic...

[2] http://www.mirlab.org/conference_papers/International_Confer...

daturkel10y ago

"Specifically, this project was born out of a yet-unpublished image deduplication framework, while attempting to identify whether duplicates were scaled versions of one another."

A humble but fun origin story for a very cool project.

0x09OP10y ago

Embarassing side-note: I've never made the mistake of trying to add an empty directory to a git repo before; if you tried and failed to build, it's been fixed.

Houshalter10y ago

You might be able to detect if downsampling is done incorrectly, since many programs just average the pixel values instead of the actual light intensity.

vinilios10y ago

Any idea if there is a similar app for sound files?

j / k navigate · click thread line to collapse

15 comments

15 comments · 9 top-level

vortico10y ago· 2 in thread

vonklaus10y ago

Interesting, is that how google rev img search works?

vortico10y ago

It's more complicated than a hash, but who knows what their proprietary engine uses. Here are some guesses. https://www.quora.com/What-is-the-algorithm-used-by-Googles-...

joshu10y ago· 2 in thread

Could you do this over a sliding window and calculate a score for regions? That would let you see if something has been composited from up scaled images.

0x09OP10y ago

Usually multiple results are a misdetection -- for example heavily compressed JPEGs will mess it up pretty badly:

    $ convert Lenna.png -quality 50 Lenna.jpg
    $ resdet Lenna.jpg
    given: 512x512
    best guess: 448x384
    all width        height
      448 (77.49%)     384 (90.19%)
      384 (66.99%)     448 (89.78%)
      320 (60.68%)     320 (72.54%)
      388 (55.92%)     256 (56.43%)
      445 (55.49%)   
      345 (55.47%)   
      242 (55.06%)

(the block-based nature of JPEG compression/quantization creates a similar zeroing effect along every 1/8 of the spectrum)

joshu10y ago

Sure. But I meant for each region, show the underlying resolution of that area. So like for FFT of a whole sample vs FFT of short periods to make a spectrogram.

mozumder10y ago· 1 in thread

mey10y ago

Not all cameras use the Bayer pattern (there are alternatives) and Canon (and I believe Nikon) are looking at ditching it entirely

http://petapixel.com/2015/10/29/canon-120mp-sensor-reportedl...

sigmar10y ago· 1 in thread

Does anyone know of something similar for automatic analysis of audio using the FFT? There are visual ways to do it, but it would be nice to have an automated script

0x09OP10y ago

Late reply but you can actually do this with the library as is. It's not designed for it so you'll have to use fftw yourself and call some borderline internal stuff, but it ought to work the same.

    float* samples = ... //load audio samples here
    int len = ...
    fftwf_plan p = fftwf_plan_r2r_1d(len,samples,samples,FFTW_REDFT10,FFTW_ESTIMATE);
    fftwf_execute(p);

    RDResolution* res;
    size_t count;
    RDMethod* m = resdet_methods();
    m->func(samples,len,1,1,&res,&count,12,m->threshold);

stevetjoa10y ago

[1] http://www.signalprocessingsociety.org/publications/periodic...

[2] http://www.mirlab.org/conference_papers/International_Confer...

daturkel10y ago

"Specifically, this project was born out of a yet-unpublished image deduplication framework, while attempting to identify whether duplicates were scaled versions of one another."

A humble but fun origin story for a very cool project.

0x09OP10y ago

Embarassing side-note: I've never made the mistake of trying to add an empty directory to a git repo before; if you tried and failed to build, it's been fixed.

Houshalter10y ago

You might be able to detect if downsampling is done incorrectly, since many programs just average the pixel values instead of the actual light intensity.

vinilios10y ago

Any idea if there is a similar app for sound files?

j / k navigate · click thread line to collapse