Relatedly, I wish I could automatically freeze and archive every single web page I visit, minus heavy media, possibly with very low quality images. I tried squid and the internet archive’s proxies, but MITM’ing myself is just slightly too annoying. There’s SingleFile[0] which does pretty much exactly what I want, ripping every single page into a self-extracting HTML+zip file, but it runs inside the browser so it adds a little delay after you navigate to a page, again slightly too annoying. Anyone have a recommendation for a seamless way to do this? Otherwise I’ll probably roll my own extension that pipes every URL to a local process that rips in the background with e.g. selenium.
I wish there were a way to run fully privileged extensions in Firefox, i.e. in the browser context instead of the page...
I might add an option in the extension for delegating to a "daemon" (based on the CLI code [1]) the capture of a page.
[1] https://github.com/gildas-lormeau/SingleFile/tree/master/cli
Been using it for a little while and it's definitely interesting although I'm not sure it backs up images as well, but could be worth a try for you.
That said, I use pinboard to save/bookmark links, and I paid for the archival account type which automatically stores the pages I save. There's a handy bookmarklet so saving a page is a one click operation.
While the civilization doesn't depend on my data, I always like to have a backup, so paying for a service to store my web archive is barely more future-proof than saving links.
Or this: https://github.com/Rhizome-Conifer/conifer
Maybe this: https://github.com/webrecorder/pywb
Or WWWOFFLE: http://www.gedanken.org.uk/software/wwwoffle/
Also pinboard.in but that's $
At work we instrument our puppeteer (headless chrome scripted) tests so that it basically sumps out the Dom and other important rendering things. I then wrote a tool that lets one step through the dump with an interactive timeline showing the current state of the page along with console events, user generated events, etc.
So the power is there if you want to get your hands really, really dirty.
I had to step away from using electron because I encountered segfaults like `Received signal 11 SEGV_MAPERR 000000000060` just on visiting cnn.com and clicking one of their nav links (without much going on otherwise, which I found kinda crazy).
Electron also only recently added support for PDFs, and it's still a little buggy. I was able to get around most of these with pdf.js.
I ended up using a combination of chrome extension for injecting stuff into webpage + rpc to a local node.js server running inside an electron app (for convenience). I only made this for personal use, so I'm ok with <arbitrary_caveat>s.
I personally save pages as plain text though so that I can use grep etc. later on. Pages with many essential graphics I also save as PDF or epub.
It isn't perfect but it is really really nice when it works.
It would really suck if I spent months accumulating data for some research, only to have the account banned or suspended, or the app shuts down.
Question: are CSV exports ok? Any other format you'd prefer?
Thanks for your feedback and interest!
I moved on after that. If this is the UX of setting up a password, I'm not up for the rest.
The sign-up page actually does include the password requirements: "The password must be at least 8 characters long and include upper and lowercase letters, numbers and symbols."
In sum: * 8 chars long minimum * include all of these: upper and lower case letters, numbers and symbols
I do realize that such a combination of characters is somewhat hard to remember compared to simpler passwords. I'll consider options to simplify all this.
I appreciate your feedback.
What do the numeric figures mean at the top of a topic card?
Things like 1b, 7b, 2.1bt and 11% and 17%
Is there somewhere that explains those? Did I miss some internet wide naming convention ??? :)
Good question, I forgot to add that to the help section :)
B = board
t = topic
b = bookmark
bt = bookmarks per topic
The percentage is the "coverage" received by a given topic in each board. If a board has 10 bookmarks and 3 of them are labeled e.g. "topicX", topicX's coverage is 30%. Totals may add up to more than 100% due to multiple topic assignments.
It helps you figure out where you stand in terms of the information you collect. I sometimes have to e.g. research and compare a number of topics within a given theme (e.g. companies in nanotechnology), and I want to make sure I do enough digging on each one.
I'm here if you have more questions!
Although 'synthesise' has broad meaning, I think organising the bookmarks this way helps to fetch our bookmarks when needed e.g. A need gap - 'I forget my web bookmarks quite often'[1] was posted on my problem validation platform which I think this tool can effectively address.
Did you have the intention of solving that problem when creating this tool? Then I would suggest improving the copy in the home page from 'Synthesise' to something on the lines of -
'Don't forget your bookmarks again, get them back when you need it'.
Also, an address bar integration which brings results from Klobie first could serve the purpose better IMO.
[1]https://needgap.com/problems/57-i-forget-my-web-bookmarks-qu...
My specific use case is doing initial, exploratory web research to work on company, industry and market analysis. This initial phase requires reviewing many sources and identifying the "important bits" that will help structure the main project. I take notes for that, at the bookmark, topic and board (theme) level. I get what you say, the wording of the one-liner maybe doesn't reflect all possible use cases, or even properly describe my own! I'll consider options for this.
The idea of address bar integration is terrific.
Thanks again for the feedback!
A couple of suggestions: I like the 'card view' of the board pages, but assuming one has a lot of tags, and many bookmarks per tag (or even many bookmarks for a single tag), I feel the 'overview' you get with the board page kind of vanishes and the individual tags take over. Something like a fixed height option for the cards that becomes scrollable would be nice. Alternatively, each tag becomes a separate page, with the bookmarks being listed once you enter the tag page instead of the board page, but not sure if you want that kind of hierarchical structure. Oh, and maybe a tighter column view might be nice!
I get what you say about boards with too many topics/bookmarks. I've been thinking of different ways to display information, so this kind of feedback is very useful!
I don't know if I can still recommend the product but for the years I used to work there I know we built an amazing product.
---
A few years ago I also made a bookmarking tool expanding links which is open source and free to use: https://fragments.me/
In my use case, it's a matter of organization. But totally get what you say, classifying things into buckets is extra work. I'll consider options to address this but, would adding a topic "later" be a temporary solution to you? Then you can edit bookmarks' topics as you please, but they would initially appear under the same card corresponding to the topic "later".
Thanks for the feedback!
javascript:void function(){var e=window,a=document,b=encodeURIComponent,c=e.open("https://archive.is/"+b(a.location)+"%26title="+b(a.title),"td_popup","left="+((e.screenX||e.screenLeft)+10)+",top="+((e.screenY||e.screenTop)+10)+",height=510px,width=550px,resizable=1,alwaysRaised=1");e.setTimeout(function(){c.focus()},300)}();Re: title: it actually is auto populated when you bookmark a webpage with the bookmarklet AND the webpage has a title. Let me know if I'm missing something here.
Re: topics: yes! You are absolutely right, I'm fixing that asap.
EDIT: Also can I get to my home page when I click on Klobie than clicking on my username? And can I tag bookmarks with topics under different boards?
What browser do you use? Once you created the bookmarklet, you can move it to your e.g. "bookmarks bar" in Chrome and it's pretty handy (that's my current set up).
I'll consider your navigation suggestion – any specific reason why you prefer to click on the logo to go back to your home page rather than the username? Any problem with the interface? Visibility?
Each board has its own set of bookmarks and topics. If you want to save a bookmark to multiple boards, you'll need to do save it to each one of them. Sorry for that – this is at least for now, a copy bookmark function is an option that I'll consider.
I'll be here if you have questions, or you may want to just send an email to the contact email (address here: https://klobie.com/help)
Thanks again!
I wonder if you could tie in with Googles related search terms to get suggested ideas to further research and fill out your board?
2 validation errors detected: Value at 'username' failed to satisfy constraint: Member must have length greater than or equal to 1; Value at 'username' failed to satisfy constraint: Member must satisfy regular expression pattern: [\p{L}\p{M}\p{S}\p{N}\p{P}]+
Thanks for reporting!
A request: Make `topics` optional or ability to set a default topic(s)
"Klo" means "toilet" in german.
not sure if this is better, though.
(Using Evernote for this)
> Diigo is a multi-tool for personal knowledge management dramatically improve your workflow and productivity easy and intuitive, yet versatile and powerful
Does the bookmarking, along with a visual canvas for editing the relationships.
Feedback welcomed :)
https://chrome.google.com/webstore/detail/blink/jgnbmndipgkk...