1) Make a Meta Smart Glasses take a photo every 2 seconds.
2) Send images to some server in the cloud.
3) Run OCR and object detection/labeling on these images.
4) Then present an app that allows searching (and chatting with) your past.
I could then ask the LLM things like:
1) Where did I left my wallet?
2) Did I get my credit card back after paying the restaurant yesterday? (ADHD things, don't ask)
3) What was written in my daughter's new tshirt today?
Bonus point if the app also records and transcribes audio so you could ask the LLM things like:
1) In the last meeting, what was the deadline that we settled on?
2) What was the phone number of that person I met in the park earlier today?
3) What was the name of the investor I met today?
Bonus bonus point if it has access to your phone calls to so it can transcribe and index what others said.
See Black Mirror episode "The Entire History of You": https://en.wikipedia.org/wiki/The_Entire_History_of_You
https://twitter.com/AlexBlechman/status/1457842724128833538
> @AlexBlechman
> Sci-Fi Author: In my book I invented the Torment Nexus as a cautionary tale
> Tech Company: At long last, we have created the Torment Nexus from classic sci-fi novel Don't Create The Torment Nexus
Remove cloud from the equation, replacing it with a properly safeguarded fully owned hardware located on your own premises - basically make it privacy-respecting, minimizing abuse potential, do a proper audit, and it becomes a super desirable system for people with neurological differences (or just stressed folks - attention and memory suffer under the stress too), a glasses that enhance memory rather than vision. Sure, not without caveats and gotchas (don’t bring it to a poker night lol), but not that cyberpunky-bad either.
a local model that can analyze the videostream/files and answer questions would be great.
A restaurant should never be taking your credit card out of your view, ever. Really, they should not even be handling it: you should be swiping it yourself at the terminal at the cash register when you pay before leaving.
I'd never seen a restaurant do that in the UK.
I think it's a relic of the times before handheld payment terminals existed.
Also, having to decide on a tip percentage directly in front of the waiter is awkward sometimes, doing it the US way avoids that. Tipping is very common in the US.
I’ve preordered them.
https://github.com/jasonjmcghee/rem
Previously submitted on HN by the dev with some decent discussion 3 months ago here:
Come join in on development!
It's MIT Licensed.
I also kicked off a cross-platform version in rust https://github.com/jasonjmcghee/xrem which is earlier in development and could use even more help.
My main and daily use case it to look back at the previous day - this is helpful for standups, retros, and so on. I skim through my days (sometimes weeks) to pick up on what I was working on - it's incredible how much "untracked" work is performed that you pick up on. Sometimes I forget who exactly I talked to about something, but just knowing the rough date I can usually find something to jog my memory.
Obviously I am biased, but still!
You can do things like find the point in history where you fixed a bug and go watch yourself debug and put in the fix.
Pretty wild. It makes a lot more sense once you experience getting the value.
Personally I hope Apple adds the feature natively to the OS at some point. They're plenty experienced to build it themselves over there, but maybe having a reference implementation will encourage them to give it a shot.
I'd love to contribute but I know nothing about Swift, mine was all in bash scripts with launchd to run them.
http://www.cs.columbia.edu/~orenl/papers/sosp07-dejaview.pdf
Abstract: As users interact with the world and their peers through their computers, it is becoming important to archive and later search the information that they have viewed. We present DejaView, a personal virtual computer recorder that provides a complete record of a desktop computing experience that a user can playback, browse, search, and revive seamlessly. DejaView records visual output, checkpoints corresponding application and file system state, and captures displayed text with contextual information to index the record. A user can then browse and search the record for any visual information that has been displayed on the desktop, and revive and interact with the desktop computing state corresponding to any point in the record. DejaView combines display, operating system, and file system virtualization to provide its functionality transparently without any modifications to applications, window systems, or operating system kernels. We have implemented DejaView and evaluated its performance on real-world desktop applications. Our results demonstrate that DejaView can provide continuous low-overhead recording without any user noticeable performance degradation, and allows browsing, search and playback of records fast enough for interactive use.
TL;DR: It's an old PhD project that requires custom patches to an ancient kernel version. So I guess you can't download it, and even if you could it wouldn't work on any system you'd want to use today:
> compsciphd on Oct 11, 2022 | parent | context | favorite | on: Linux NILFS file system: automatic continuous snap...
> we used NILFS 15 years ago in dejaview - https://www.cs.columbia.edu/~nieh/pubs/sosp2007_dejaview.pdf
> We combined nilfs + our process snapshotting tech (we tried to mainline it, but it didn't go, but many of the concepts ended up in CRIU though) + our remote display + screen reading tech (i.e. normal APIs) to create an environment that could record everything you ever saw visually and textually. enable you to search it and enable you to recreate the state as it was at that time with non noticeable interruption to the user (processes downtime was like 0.02s).
https://news.ycombinator.com/item?id=33165519
> compsciphd on Oct 12, 2022 | parent | next [–]
> sadly (as with much work form phd students like I was), the closest one could get to it today is trying to duplicate it. i.e. combining criu with nilfs (but a lot of the work that we did to get process downtime to minimal numbers requires being in kernel, as described in paper) and unsure criu can do it.
> In addition our screenrecording mechanism was our own "proprietary" (not really proprietary as fully described in research papers, but also not a standard) and something that was built as an X display driver 15 years ago (so not directly usable today even if code is available). Could probably duplicate it with vnc based screencasting. vnc didn't work for us as we needed better performance (i.e. it was built to demonstrate remote display of video and games and there was no real remote audio setup back then so we had to create our own).
> the "text" search just used gnome's accessible API much like a screenreader would do (with a bit of per application optimizations as can filter out things like menus and the like, primarily was to dump text out of terminals, firefox and perhaps open office and maybe even a pdf reader if memory serves me correctly, but a long time ago).
I've been looking into it myself though, mostly for forensic concerns (capturing state/evidence in a way that's harder to forge than screenshots). Hopefully, running the desktop environment through VNC and then running `criu dump --leave-stopped --prev-images-dir …` immediately followed by `mkcp --snapshot …` (or `btrfs subvolume snapshot …` would also work, I guess) would be enough for basic functionality.
This one works : https://www.cs.columbia.edu/~nieh/pubs/sosp2007_dejaview.pdf
This one’s broken: https://www.cs.columbia.edu/~orenl/papers/sosp07-dejaview.pd...
When I search for “dejaview” in the main index, I get the same broken link in the search results[0]. At first I thought they’d changed the URL structure (/pubs/ to /papers/), but if you visit [1] it works, but [2] doesn’t work. I guess “orenl” isn’t a member of the faculty anymore, so they tore down their page and removed all the associated resources.
[0] https://www.cs.columbia.edu/g-search/?q=dejaview#gsc.tab=0&g...
Excellent product for a consultant like myself to have on hand to document everything happening on their machine. The ability to change the screenshot interval and add notes along the timeline are real power tools for someone involved in multiple projects every day. You can secure your time record snapshots with a password. Playback at multiple speeds so you can jog your memory by browsing the timeline to the exact time when you flipped to something else. Classify by project, client, etc.
Really a great bit of software.
I once found one of my invoiced time periods being challenged during a regular audit of one of my client's operations. I frequently billed long hours since I worked long hours on 24-hour call (remote oil and gas data processing and support of international operations). The auditor called and requested anything I had that could support a particular time period where I billed "excessive" hours (>20 consecutive). I told them I would get them everything that I had and asked whether they were interested in any other dates or time periods or work for specific projects. They narrowed it to one single invoice entry so that made it simple for me.
Thanks to TimeSnapper I was able to produce a GIF movie of the screenshots (I always used a 1 minute snap interval to allow fine-grained operations review) during the requested interval that documented everything happening on screen and I provided a PDF of the notes dumped from that interval documenting phone calls taken, etc. I also sent copies of my handwritten notes I always maintain as primary memory-jogging tool. In the return note to the auditor I asked if they were interested in any other dates or time periods and offered to send an archive on their request.
The auditor called me up and we talked for a while and they thanked me for the detailed records and told me they wouldn't need anything else.
It really helps that my wife is a CPA/Auditor/Fraud Examiner. Tools like TimeSnapper Pro helped me in my business to maintain excellent records about day-to-day operations that made it simple for me to invoice clients for all the time that I spent on their behalf.
Thanks /u/LeonB
Prior to TimeSnapper for Mac I ran my own little script, and a companion one for Linux. Now I have TimeSnapper on my Macbook Pro too. And if you ever create a Linux version (or want someone to create a Linux version), let me know. I'd be happy to beta test.
I've lost count of the number of times that TimeSnapper has saved my arse, either with accidentally deleted data, finding an obscure web page, proving I did something on a specific date/time, or that something happened in the way I say it did, or recovering the keys to a small amount of Ethereum and Dogecoin stored in a wallet.
Thank you for TimeSnapper. It has been interesting to use it for that length of time.
"Four weeks work in 9 minutes" captured by TimeSnapper where I create a video game (or three) in October of 2012 http://www.otakunozoku.com/video/working.flv
Sorry, it's an flv, along with an onion on your belt, it was the fashion back in the day
Surprised to see that "Maintain aspect ratio" is only there in the pro version. Maintaining the aspect ratio should be a basic feature.
I concluded that in case this is a viable approach, Microsoft or Apple will built that into their OS natively as part of a copilot that remembers everything and comes to assist the user with the knowlede.
My screen history was not as advanced as the app mentioned here though. And I didn't use it myself.
I’ve long thought that ideally an auto-archiving transparent proxy would be a more elegant approach.
I distinctly remember seeing it somewhere (I think it was here), but I can't find what the app was called, despite all teh gooling.
I made one called flyonthewall, you can upload a m4a file and it gives back transcription from openAI whisper model. Basic setup (no accounts right now) that I use to transcribe plus summarize my audio notes from my dictaphone.
Original Savant source? https://bitbucket.org/theluxury/savant/src/master/
Videos of how the later Atlas Recall worked: https://www.youtube.com/@atlasinformatics3316/videos
I know my screen can be checked at my current client, but I also know the product owners don't really care as long as the work is getting done on time (or at least they haven't bugged me about it since I've starting working with them a year ago and have given very positive reviews about me to my company).
I think employers would be more interested in basic things like “did this person login their computer today, between what times were they active, how many hours in the day did they type at least 100 words?” … or things like “here’s a list of people on PTO today, which people not on this list haven’t typed on their keyboard today?”
That sort of data would be easy to collect - keyloggers have been around forever, doesn’t trigger MacOS to show the “recording screen” notification, and the data is easier to aggregate and view across a large number of computers.
I think the fact that keylogging didn’t explode as a method of tracking productivity during Covid WFH probably means we’re safe from screen recording for the foreseeable future.
There's a huge miss in this implementation though: 95% of the time what is on screen is web-browsing. So to go from nicely formatted markup with title tags -> video -> OCR is clearly missing am ore obvious path (either a proxy or browser extension).
I've often thought that screen readers might be better off now by attempting this visual approach. It used to not be possible, now it may be easier than relying on accessible markup.
And yes, most of my screen time is also web-browsing. Currently, I can use the web page's Windows title as a reference, which provides quite a bit of information. To enhance this feature, it might consider using like Chromedriver to gather more details, such as web links and page text, similar to what 'Rewind' offers.
I made it, for full disclosure.
This is tremendously helpful for people like me who forget stuff often.
I would propose you are both absolutely correct.
Copying ffmpeg to C:\Windows\System32 doesn't seem to be the correct way to install it.
This is the windows way. In the early days (Windows 95/98) it was indeed common practice to put shared dlls and exes in C:\Windows or C:\Windows\System32, but without central oversight this was a nightmare: version conflicts were rampant, and uninstallers didn't know what could be safely removed. Everyone switched to installing everything into the program's install dir, and the world was much better off.
Some poweruser tooling gets fancy by checking if ffmpeg is installed systemwide, and if not downloads it to the programs appdata folder. But that exposes you to versioning issues if that version is too old or too new
> Please enable JavaScript to continue.
I run NoScript, and the way this block is constructed makes it impossible to actually enable JavaScript to run the site.
Please don't redirect to a static page, because that page doesn't have any of the JavaScript present to enable on a case-by-case basis, the the redirect happens so fast I have no chance to enable JavaScript on the main page.
Just enabling for notion.so itself is not sufficient to bypass the block.
Also I'm surprised html even allows for non-JS redirects.
It seems to create a huge risk for your data though. Any ideas about how you might go about securing against the application itself doing something untoward?
Anyway, in my lab environment where I had this running .. over a decade ago now I guess .. I set up scripts to automatically cryofreeze the daemon processes I was writing, so I could indeed reset to the state in time, of interest during testing. This lab got re-comissioned for other things, and a needed kernel upgrade broke cryofreeze, and I never got back to it - but I have often wondered about it, as a temporal interface to some things would make for a whole new world of useful UI and other interation paradigms to explore.
I remember there was something like this possible on WANG and Tandem systems back in the 80's, and it always has kind of piqued my interest as to why this isn't still considered a thing. Well, I guess syscall complexity has vastly changed since WANG and Tandem were around, lol.
Anyway, I'd love to be able to do this on Linux again - I keep thinking to catch up with the state of the art of process freezing/unfreezing, so this has motivated me to do this - although I do confess that, as principally a Lua/C developer in my chosen market (embedded), I attain this re-playability in other ways (state sync'ing) which also allows me to move processes to other systems, relatively smoothly, also.
Still, would love to have this for normal Linux processes, out of the box. Am I ignoring an obvious way to do this?
> Please refresh and try again or.
