A few years ago, I left my cushy big tech job to make hardware.
And made the device I always wanted - an automatic piano recorder!
I usually play piano improvisationally, and manually hitting record never meshed well with that. But there are always moments I wish I recorded, and now they are!
Hopefully it scratches a similar itch for some of you as well!
A few of the tech details: * built on an esp32-s3 * custom injection molded enclosure * BLE comms, sd card storage, DS1302 RTC * android & ios apps with Flutter * Shadertoy vfx support for video sharing
- Chip
Your project got me thinking - here's one idea: Windows should get MIDI 2.0 support soon, incl. non-blocking MIDI reading if I understood correctly. That should make it possible to create a small background application that records all incoming MIDI from all (or chosen) connected MIDI devices. It would work very much like your recorder and could share the same mobile app?
This I would be interested in. Since it's a software only solution, it could be cheaper and lower entry barrier.
See announcement here: https://devblogs.microsoft.com/windows-music-dev/windows-mid...
Since most sustain pedals come through as a CC message I think it's supported.
This might be the first Show HN that I insta-purchased after reading your landing page. The mobile interface looks extraordinarily well thought-out.
This is an absolutely phenomenal use of the espressif esp32s3 in a hardware that I will likely use daily for the rest of my life.
Kudos!
You should have seen the original UI concepts I made, they were terrible!
Seems the ESP32 can to Speex encoding so I guess Chip could integrate this as well. If he manages to earn money on this.
I was about to order but frankly it does not feel good to pay about the same amount of shipping (to Finland) as the product costs.
If you can do anything about it, I would be happy to order. 90EUR for shipping is just too much, 20-30'ish would be reasonable.
Yes I have a lot to share about the production process. Blog posts are planned! It was quite fun!
Works basically like this: You collect couple of shipments to one bigger package. Then you send that bigger package to Asendia logistics centre and they will send those individual packages to users.
This would also possibly widen the market for the product a little. Playing non-piano instrument sounds, even very technical synthy stuff, on a keyboard should “just work” assuming that all outgoing MIDI data like Program Changes and Control Changes are recorded and replayed. The further you get away from pure piano, the less likely the result will sound right on anything other than the original keyboard/piano.
Incidentally I'm curious if SysEx are recorded. Sometimes those are also used. For example some of the effect controls on certain Yamaha products use SysEx for historical reasons.
If you do add playback over MIDI you may want to familiarise yourself with some of the “reset” mechanisms to avoid hanging notes and other such problems, and make the app send them at appropriate moments. All Notes Off, All Sounds Off and Reset All Controllers are particularly useful “special” Control Changes.
To be clear it isn't in any way a deal breaker as the Zynthian's CPU isn't quite good enough to run Pianoteq in the best quality anyway, and moving some MIDI off to a DAW later to create a higher quality version seems like a great option — just it'd be really cool to be able to play things back the same way I played them initially :)
Just needs to let me "listen" through the midi out instead of the phone.
Caveat if course is, that I'd need to remember the synthesizer/settings/presets to have it sound the same again, but maybe half the fun to guess a bit and get happy accidents.
Why would you start with that? Particularly since some of the best "digital pianos" are actually just MIDI keyboard controllers (weighted and all that) and don't actually generate any audio themselves?
My understanding from OP's product is that there is an inherent assumption that this is targeted towards people dabbling on standalone digital pianos that also typically have built-in synthesis/sound modules.
Anyway, I'm just giving my reasons for why wouldn't you start with a software approach to answer your question. Putting all that aside, OP's solution is really cool and, independent of the decision to go custom hardware, it seems like a great product solution for the problem it's solving.
Only way this works in software is if you dedicate a device to always record and audio processing doesn't provide isolation. Ambient sounds may get picked up or loud noises could drown out your playing.
This item works, because it is hardware. I also wonder how this would work connected to something like the sonuus G2M V3. Would be nice if the audio processing was solved by someone else.
It also has an open API, that's pretty simple. So you could integrate it into other apps or open source software.
Also: Less space than a Nomad.
Yes there are some limited times you might want to turn off recording. I've given this some thought. For now, unplugging is the simple answer.
Actual sounds are coming via Pianoteq though.
Sample: https://www.youtube.com/watch?v=PSHis77Y1f8
I use OBS to combine all the various elements.
Btw, for anyone that purchases, there is also a (very new) subreddit!
For me, I'm finding that firmware development and app development is an absolute pain compared to mechanical design and PCB design. What was your experience like?
For the firmware, I would have gone with Linux + a higher level language, probably Golang, because my iteration loop was almost 2 minutes w/ C & esp32. Way too long.
But at least it made the product very power efficient, and a bit cheaper.
Btw, you should follow the hot reload thread on Github for ESP-IDF: https://github.com/espressif/esp-idf/issues/12642
fortunately it turned out that the s3 hardware (i've got a cardputer) isn't that crazy, and it is relatively easy, if almost completely untrodden, to program much of it from scratch (which is the familiar way to do embedded for me).
my iteration loop is ~30ms for a small program (i download directly to ram so as not to degrade flash).
people are now actively working on reverse engineering wifi, which so far has been the major reason to use espressif's software stack (their driver runs under freertos).
About efficiency - ESP chugs gobs of power, comparatively (something like 50mA average, and that's with all the relevant power saving features enabled). By comparison, an nRF or an STM32W0 is single digit mA.
I ordered one to use secretly. My dad’s 90 and was a working musician since he was 15 or so. He’s always wanted to make recordings of his playing but gets distracted by the technology, even something relatively plug and play like Garage Band. I’m looking forward to plugging this in and just capturing his playing without distraction.
Should be pretty straightforward! I'd want a decent UI for it!
Could there be a Create Bookmark Now button in the app as a fallback?
**
My current setup in comparison :
-Had to grab the USB-B cable from the printer
-Laptop on battery balancing on the corner of the digital piano
-Launching a custom script piano.sh (I remember that the initial install process on ubuntu was painful) (The script (3 lines) : qsynth& aconnect 24 129 rosegarden& )
-Then hit record on rosegarden
-It works I can get the music out of my head
-Then I fight with the UI to select the time position to be able to playback
-Then I select another track to record another layer on top while the playback is running, I misconfigured the recording and it recorded over my previous track.
-Then I have lost my music.
-Then I test what happens when I change the instruments on the piano : rosegarden doesn't record the instrument change and the soundfounts are different than what my piano have so it doesn't sound exactly the same
-Then I close rosegarden, remember why I hate it, plug a jack in the line-in start audacity, hit record, play music to calm me down.
-Save wav file in a lossless compression format. Store it in a file somewhere where it most likely won't ever be played again, telling myself in a few years I'll just run a script that will transcribe it and organise it directly.
**
My setup is awful from a cold start, but gets better with practice when I remember how to use the UI. The ability to playback an existing track while I record a new track is what I used it for. And the metronome. The main problem with midi is that the sound fonts are not exactly the same.
I probably won't buy it though, I don't want yet another app. I am not sure if it will work in 10 years. The sound is not exactly what I played. It something that should have been an option in the piano and not a $99 option.
> I am not sure if it will work in 10 years.
This was a big concern of mine too. That's why as much of Jamcorder as possible is open standards.
There is also a local web interface, http://jamcorder.local, that is fully extensible by the user. And the BLE and Wifi APIs are not locked down. You can implement your own app.
The biggest limiting factor IMO, are Wifi and BLE still going to be in common use in 30 years? That's hard to predict. But the SD card will still be there. Its a tough problem.
I think that's about as good as you can do, short of putting a screen on it.
----
There's even a feature identical UART interface for the device. Surely that will still work in 30 years.
I suppose an AI could be trained to look for errors and act as a tutor, eg recommending practice pieces to counter bad habits or to encourage more competent play - like, your relative values are poor, practice this 'metronome' piece; or, you only play bar chords with your left hand, try this simple piece to focus on left-hand arpeggios.
Would have loved to not have lost so many improvisations, and consciously recording every time before you start playing is too much hassle.
With that said, I'm grateful for the mechanical stability of it, and also the reliability of the interface.. Should not be taken for granted. It has buttons, not a touchscreen for example (Kawai digitals were sadly out because of this.)
However, the internal capacity is fairly limited and there's some tolerable yet a delay after the recording to save the buffer on the flash.
I don't think there's a ready way to copy such internal recording from the piano onto some external medium.
This device may be a nice 'upgrade' for such digital pianos. It'd be nice if the recordings can also be played for a selected hand, so that it could be used in learning.
I assume I can just drop this guy in between my midi controller and my primary interface since it has MIDI I/O, yeah? Does it pass through?
Most interesting!
Did you write your own support for USB-MIDI or are you using some ready-made library?
I remember once hacking on my own project with USB-MIDI support and it was a hassle. I was constantly missing MIDI messages. Had to build a "panic button" in to force "stop all notes".
Although my project was using SAMD21 (Cortex M0+), so possibly it wasn't fast enough for the use case.
modern ESP-IDF: https://github.com/espressif/esp-idf/pull/12566
versus 3 years ago: https://github.com/chipweinberger/xesp-usbh
I had a Fantom X6 and one feature I loved about it is that it was always recording in the same manner as this, you could just push the "Skip Back Sampling" button and then save or do whatever with it. It was also good because the audio samples were synced to the midi clock. A cool feature.
Up until now I've only known of some DAWs & custom raspberry-pi like solutions.
Though the Fantom X6, you have to push 'record' afterwards. Still cool, though!
Thanks for sharing!
It'd be so easy to do a version of the "infamous Dropbox comment" on this ("you can already build such a system yourself quite trivially by getting a MIDI cable, an audio interface, and a raspberry PI...") but of course what you have is exactly a sort of Dropbox Of MIDI here where it Just Works™ and backs up all your music automatically with no hassle.
So relatable.
Congrats, looks like a great product. I just ordered one for my piano-player buddy for Christmas.
It can also be a mood killer when rehearsing with a band. Everyone is messing about and having fun, then suddenly REC ON! and everyone is almost dead serious in performance mode.
Is that to be inline with other IoT devices UX, or is there a technical reason like esp devices cannot be in AP mode unless initialized at boot?
In theory it should be fixable now.
- Can the out be used as a thru MIDI port? So I could just jam the device into my workflow without having to acquire a new MIDI splitter?
- What of MIDI it captures, exactly? I understand note on/off, velocity and such is obviously captured, but what about CC values and everything else? Would be wonderful to be able to hook this up to synths that have more controls, and be able to capture those values too
All midi messages are captured, on all 16 channels.
Having to lug my laptop out, boot'up Bitwig, connect it to the piano, press record completely takes me out of the flow.
With this, I can flip on my Kawai and start banging away, then pull out the impromptu motifs for later refinement.
are you using flutter and blue plus ??? because I see your post and someone on reddit asking about bluetooth then I get rerouted on same web page lol
its crazy coincidence because I open both page at the same time
btw check out http://reddit.com/r/jamcorder for ongoing discussions.
Looks like it does.
Is it recording 25,000 hours of actual audio which it analyzes, or is it recording midi data?
just remeasured it to double check.
Congrats on the launch and bravo on such a well-polished everything - product, UI, website, etc. Very impressive.
If you want historical, it's also exposed over the web interface and wifi API.
just curious. Thanks.
This is a really cool example of ambient technology. Typically when people talk about ambient technology they're talking about something like an e-ink display that is pushing information to you, but in a way that doesn't require interactivity and isn't screaming for your attention. This is a little different in that it's always _receiving_ information from you without any need for interaction or maintenance, except on your terms.
The interaction model is pretty clever too. Since it's collecting data from the instrument they've found a way to cue the device to perform an action without the user needing to open an app. (black keys to bookmark) There is an app of course, but it connects directly to the device, with no annoying setup requirements. I've seen this same approach with several other devices - Xbloom coffee maker, Combustion thermometers, Week Aqua lights - it works really well. I'm understating it. It's astounding how pleasant it is to use devices like this.
As hardware continues to improve I expect this will be the default mode for pretty much every new technology appliance. Ambient operation, local data, local app, with cloud and accounts as _options_ to extend functionality if it's necessary.
From the FAQ it sounds like you have plans to extend the search feature set - excited to see what's in store! I feel like a similar parallel here might be the search functionality in Google Photos. If you can do for MIDI recordings what they did for photos that would be huge.
[1] https://github.com/seletskiy/dotfiles/blob/master/bin/piano
I'll buy it just because of this.
If this launched 5 years ago, I would have immediately purchased it! At the time I was using pencil & paper, Sibelius, or my phone to record improvisations and all of them were very poor solutions.
I've reached the point where I use a DAW (Reaper) + MIDI keyboard + sound libraries. Conveniently, Reaper can record & display everything as editable MIDI output. Pencil + staff paper can be great if you're slowly exploring something though.
There are MIDI interfaces for phones. It is astonishing that there isn't an app which it's the checkboxes.
For the room
The worlds smallest midi synthesizer https://www.youtube.com/watch?v=KTvVS8guBsY
There's something so whimsical about this kind of tech. The cost to store the midi is effectively zero, it would be so interesting to see it installed within a keyboard - a journal chronicling the player's progress over decades, and everywhere the keyboard has been. I could see something like this being included in the next OP-1
Indexing. The problem you face after recording too much.
If that's not enough, stay tuned! more tools like grouping by chords, songs, melodies, faster navigation, & search, are planned!
Probably needs to have a song database, so it can index by song name.
I have a very minor feature request: MIDI loops! Add a button on top of the device: Press once to record, press again to stop. Quantize on/off available via the app.
Awesome stuff!!!
Would love to see anything of the visualizer. Understand it’s beta, but curious what direction it would go
And I love the name.
