Most simulated pianos these days add the extra harmonics when the sustain pedal is pressed, but very few seem to get the harmonics from the dampers being lifted per-key; PianoTeq gets this right.
I wonder what the last mile is or whether acoustic modeling will always be stuck in a kind of ‘uncanny valley for sound’.
I think you're touching on the overall concept of harmonic and enharmonic overtones in sound synthesis.
Spent a summer going down the rabbit hole of tuning it myself. I found the Entropy Piano Tuner app to be quite helpful. Took me weeks to get it tuned to where I liked it.
My digital Yamaha doesn't even come close to capturing the resonance you get on a real piano. I find it hard to play digital pianos for anything other than simple melodies.
Entropy website: http://piano-tuner.org/
(Though in practice both of my pianos have been comparatively neglected since I got a Kawai DX1900 drawbar organ a year ago, which I think is rather amusing. Analogue electronics are really fun, the wear and failure modes are much more interesting than digital electronics like they’d use if making this kind of thing these days.)
I should try Pianoteq and Organteq.
Electronic speakers cant do this 100% generally speaking, especially when its not some really high-end audio installation. (I've been on projects where I was just in awe of what sound engineers can do with spaces) The sound would have to make a reflection or two before it actually hits all the same surfaces?
But most people are listening to music in a comparably humble setup and acoustic environment.
I am not an audio engineer, but this is my thought.
But when I was learning FM synthesis (one of the harder ones), I found it incredibly useful to go through the exercise of trying to make a synthesizer sound like a convincing gong, or reed instrument, or guitar string. In fact, I don't think FM really clicked until I did that.
A lot of (non-MIDI) computer music is either drones, granular clouds, or percussive blips.
There's an under-explored area of articulation and musical detail between those sounds and warm, mushy analog.
The DX7 was a good first take at that. But today you can have hundreds of oscillators stacked and patched in any configuration you want instead of just six.
https://magenta.tensorflow.org/ddsp-vst
https://magenta.tensorflow.org/ddsp
I doesn't sound realistic but it is expressive in an unusual way.
Perhaps that's because it's quite difficult to do that with synthesizers? Because once you start using samples, you loss the ability to play and modulate without it sounding like a sample?
I guess my counterpoint would be the massive popularity of Omnisphere, which combines a sophisticated sample-based synthesizer with good filters and a few other types of synthesis. Or would you consider this "organic" but not "realistic/acoustic"?
I used to own a Korg Z1, which had one of the first physical modeling synthesis engines. While it could produce relatively realistic acoustic sounds, that is not what people ended up using these types of synthesizers for. Unlike a physical acoustic instrument, where the basic geometry, material properties, and other parameters are fixed, most parameters of the synthesized physical models could be dynamically modulated and manipulated like any other synthesis parameter.
Modulating the fundamental physics properties of an acoustic instrument at audio rates produces some really interesting timbres and effects that are not reproducible using any other type of synthesis or (obviously) acoustic instruments. Consequently, no one ended up using them for realistic acoustic sounds; it was much more interesting to use the synthesis engine to do physically impossible manipulation of the acoustic model to generate novel sounds.
These days almost everything in a synthesizer engine is based on modeling due to the inexorable increase in available computing power. Nonetheless, there is still considerable convenience and economy in using acoustic instruments or samples for many purposes. Just because we can create incredibly detailed and realistic physical models doesn't mean it is worth the effort and they often have terrible UI. I could see this as being something where AI could do a lot.
Omnisphere is a good example of where the term "sample-based" isn't entirely accurate (ignoring its modeled oscillators). While it does have a large and excellent sample library, many of the oscillator engines use the samples as spectral feedstock instead of as a sound to be played per se. The raw sample is not identifiable even though it imparts a characteristic quality on how the oscillator sounds. (Omnisphere is also massively popular because it is an excellent synthesis engine with an unusually good ease-of-use to power ratio. Still one of my all-time favorites.)
I think this article is more geared towards pure synthesis when you are using very few primary harmonics. Like stacked sine waves or something. They are impossible to eq because the frequencies are too pure and unnatural so you have to dirty them up.
Personally, I'd say physical modeling still has a lot of unexplored potential, but that doesn't seem to be a popular opinion.
I could learn a new synth or I could use the Spitfire Labs BBC Symphony Orchestra pack and spend a lot of takes playing with macro wheels and maybe some adjusting after to try to improve articulation to something more natural sounding. It would by a bear live but in the studio its easy.|
The synth would offer more customization but the sample one is good enough for most things, especially in a small/home studio setting.
Edit - from the comments I found this paper which looks promising
https://ccrma.stanford.edu/~cc/vox/smac2013som/upho2013.pdf
Which also leads me to this paper by Chowning himeself from 1980 titled 'Computer synthesis of the singing voice' which uses FM synthesis.
https://www.speech.kth.se/music/publications/kma/papers/kma2...
However, is that a continuation of synthesizers or not?
* partials? Is that another word for harmonics?
* “Box tone”?
* Breakpoint synthesis? ChatGPT suggests this is a kind of interpolation but I don’t quite get it.
Any general references would be really appreciated. Thank you!
Still curious about the others.
Breakpoint synthesis https://nathan.ho.name/posts/nonstandard-oscillators/
Box tone isn't a term I'm familiar with, but from context the author seems to be talking about the sound contributed by an instrument's soundboard or cabinet. EDIT: Another possibility, https://gearspace.com/board/mastering-forum/1401491-box-tone...
You could stick one (or two) transducers to a guitar or piano and then mic it up, and get the sound of the body of that instrument and parasympathetic resonance of any undamped strings.
The body is quite easily modelled with an impulse response, but parasympathetic resonance in DSP is still not great
Speakers, themselves, have an absurdly broad range of nonlinearity, even within a single speaker, the way it behaves at 10% power and the way it behaves at 90% power can be drastically different. That makes them fairly difficult to model. Impulse responses are the soup de jour, and do a good job, but, they only really model a slice of what a speaker can do, ie, a given amplitude.
I struggled for quite a while with getting bass to sound right on different systems and pushing the bass into mono helped a ton.
Also consider the length of the waves. If you're playing on a system in a club that has the subs 50 feet apart, you're going to get weird nodes from comb filtering and the bass will just disappear in certain places in the room.
I'm not aware of stereo bass having anything to do with vinyl, the issue is overall bass volume.
Most of the other ideas are pretty good and can be summed up as follows:
Humanize/randomize pitch, velocity, and rhythm
Use distortion to add extra harmonics
Don't use a flute patch 2 octaves below C or a contrabass patch 4 octaves above C, respect the "true" frequency range of the patch
Use appropriate panning
Use reverb
(Actually I disagree with the idea of using many different reverbs. Your ears will pick up on it and it will sound unnatural. IF you do this, turn up the reverb until you can hear it, and then dial it all the way back until it seems to disappear.)
If you're going for realism, then yes. If not - push the ranges, see what happens.
>(Actually I disagree with the idea of using many different reverbs. Your ears will pick up on it and it will sound unnatural. IF you do this, turn up the reverb until you can hear it, and then dial it all the way back until it seems to disappear.)
I believe it was Zappa, trying to rescue his LSO recordings, so he placed each section in a different 'atmosphere'. Sorry, on mobile but I'm sure it's mentioned in (at least one of) his book(s).
It has to do with the needle coming out of the groove if low frequencies are out of phase. I agree with your point though. The lower a frequency gets, the less directional it gets. So, bass sounds in stereo are even more pointless for most music.
FWIW, the Roland D50 did something similar, and used PCM samples to model the attack/transients... you could then use subtractive synthesis for the body of the sound. I would guess Pianoteq's key/striker modelling is a lot subtler!
One problem with that is that good microphones are expensive, likewise, but less so, good speakers.
One of the wonders about modern electronic music is the low prices of digital audio processing (i.e. computers) equipment.
That and the general geek points for making something sound like an acoustic instrument.
https://www.youtube.com/watch?v=5NBm0CdJ8x0
Got to recommend his Sim-Cimbalom instrument too... he created and released this amazing synth 12 days after I suggested it!
https://www.youtube.com/watch?v=ALKOXqQXyTs
Many of his instruments provide little details along the lines of those in the article (see his narrated YouTube vids), all freely-available here, most-recent at the bottom:
I definitely agree with 2,3 that clippy hard transients are cool (clipping is in general a bit underrated as a creative synthesis tool).
From my general experience playing and designing with the 'classical' synth approaches (additive, subtractive, FM/PM) nice results are the cumulative result of many tiny hacks of the kind listed, well-judged micro-aberrations.
The spice makes the dish, that is.
This makes design and testing patches a somewhat infinite, fiddly task. Or endless fun, depending on your point of view.
Obviously, if you want to productise your thing - ie you'd like other people to use it - this you now have a control problem: bake in dozens of parameters such that 'it just works' or make them all available and let the user figure it out themselves.
I tried recording a rendition of Hans Zimmer's memoirs of a geisha main theme violin solo (scroll down to the youtube clip of the geisha).
The pitch being fully continuous and in control of the player injects incredible amounts of "realism" and "violin-ness" to the sound. The IR of violin body i added in this example is crude - BUT - it illustrates my point really nicely: https://modwiggler.com/forum/viewtopic.php?t=68150&start=100
I think both the attack pitch envelope is crucial to get this right as well as the pitch bends induced by the player.
An even better illustration is in an even cruder synth i own - Unisyn's Pitch to CV converter has a VERY crude tri/square/saw wave - and the the pitch + volume - is the main carrier of expression - this is an example of me playing a bamboo flute into a mic - and having it converted to a saw wave synth patch through some reverb added in post - the flutiness is remarkable here imo:
https://soundcloud.com/khaleeji/unisyn-flute-and-vocal-recor...
I have to shout out Fricko modules - in Euro being really cool forward thinking modules which aim to inject some novel building blocks for building acoustic sounds: https://fricko.home.blog/
Especially if everything is tuned so subtle that it’s just barely noticable.
