FilterHN

When Fast Fourier Transform Meets Transformer for Image Restoration (2024)

61 points

by teleforce

2 days ago

| past

| 5 comments

| github.com

| HN

▲

jongala

37 minutes ago

[-]

Relatedly, Marcin Wichary wrote a nice post about using FFT to remove moiré and halftone effects when scanning images that were printed with halftones.

It's from 2021: Moiré no More (https://newsletter.shifthappens.site/archive/moire-no-more/).

▲

TimorousBestie

4 hours ago

[-]

There have been some interesting advances in trying to add spectral information to the data that a learning architecture has at its disposal, but there are a couple roadblocks that I don’t think have been solved yet.

1. Complex-valued NNs are not an easy generalization of real ones.

2. A localization in one domain implies non-local behavior in the other (this is the Fourier uncertainty principle).

Fourier Neural Operators (FNOs) come close to what I want to see in this area but since they enforce sparsity in the spectral domain their application is necessarily limited.

▲

FuckButtons

4 hours ago

[-]

I do wonder if a wavelet transform might be better.

▲

TimorousBestie

2 hours ago

[-]

I think one can do better with a wavelet, shearlet, or curvelet transform that is adapted to the problem domain at hand. But the uncertainty principle still haunts those transforms, and anyway the goal is to be domain-agile.

▲

waynecochran

2 hours ago

[-]

Was there a conclusion?

▲

sorenjan

4 hours ago

[-]

See also: CosAE: Learnable Fourier Series for Image Restoration (2024)

https://sifeiliu.net/CosAE-page/

▲

gryfft

5 hours ago

[-]

[2024]