"You should realize at the outset that while knowing about the internals of Mathematica may be of intellectual interest, it is usually much less important in practice than you might at first suppose. Indeed, in almost all practical uses of Mathematica, issues about how Mathematica works inside turn out to be largely irrelevant. Particularly in more advanced applications of Mathematica, it may sometimes seem worthwhile to try to analyze internal algorithms in order to predict which way of doing a given computation will be the most efficient. But most often the analyses will not be worthwhile. For the internals of Mathematica are quite complicated."
Reference: http://reference.wolfram.com/language/tutorial/WhyYouDoNotUs...
For comparison, if you want to audit the Sage Math algorithms that your research depends upon, all you need to do is fire up a text editor (or browse their github). And you won't find any statement in the Sage Math docs telling you not to bother because you're too dumb to understand what you're reading anyway.
In math, how you came by the results you came by is always relevant. Don't tell mathematicians they don't need to know that. It's their job to know that.
and honestly, it seems like diving into sage may not be as trivial as you make it sound. is it not a massive glue of many different languages and implementations?
He (and his group) developed CHOLMOD and UMFPACK and other sparse solvers used everywhere. Basically, when you type A/b in Matlab, it calls his code.
It was an incredibly challenging task going from sort-of/kind-of being able to solve linear systems to where we are today. Hardly anybody thinks about it. Again, you just type A/b, even when A is poorly conditioned. You can write a crappy solver in less than 100 lines of code, but if you read his papers, building a rock-solid solver was a very difficult task.
Unfortunately this kind of work is important, but pretty thankless.
[1] http://www.mathworks.com/help/matlab/ref/ldl.html?requestedD...
But in this case, and in the context of the OP, it was Davis's oversight and possibly tangible sacrifices (I don't know the details of his career) that made it possible.
Each time we use some numerical methods and it Just Works, we should mumble a tiny prayer of thanks to folks like these, and the writers of BLAS, LAPACK, ARPACK, and many others that provide the backbone of an incredible amount of work.
x = A\b gives the solution to Ax=b, and x=b/A gives the solution to xA=b. But when would you type A/b ?
Mathematicians who write software are no more likely to be "truly making the world better" than those who prove theorems and teach hundreds or thousands of students every year.
I'm not saying anything about Stein here. He's a rare mathematician: strong on theory and practice, a passionate advocate for his causes, and a respected teacher. But there is a danger of missing the real point here -- most good things don't get funded or recognized. This has gone on for centuries. The funding situation now in the US (for both theory and practice) is better than it has been almost everywhere for almost all time.
How can this be true in the case of a mathematician who writes something that many/most of the others use?
--
Open source basic infrastructure -- everything from Sage to OpenSSL -- suffers from a market failure.
We rely on these foundational projects for billions of dollars a year in commerce, but they often get minimal funding and are supported by semi-broke volunteers working in anonymity.
* GPG is maintained by one guy, who was about to give up before a few people threw coins in his tip jar after this story a year ago: https://news.ycombinator.com/item?id=9003791
* OpenSSL was comically underfunded and underappreciated until Heartbleed happened and people remembered how much it matters
Stein's story is powerful and shows how neither traditional companies nor universities help here. In the world of math software:
* The companies created a bunch of closed-source walled gardens (Mathematica, Matlab, etc).
* The universities were unwilling to support free and open tools. They gave tenure and support only for authors of research papers, not tools, no matter how useful or widely deployed.
Even the guy who made NumPy and SciPy didn't get recognition for it---wtf.
I think that these new, independent organizations with rich patrons can fill in the gap. Organizations like YC Research, Project Zero, and Canonical.
We need more of them.
Academia values mindless papers over actual results.
I'd prefer it if the granting agencies supported this sort of software infrastructure directly, but, lacking that, a company is a way to hire people to tackle some of the weaknesses of Sage, whether they be in its core functionality or its UI.
Anyway, my point is that even if you're in a niche, if you are the clear expert in that niche (say if you created the framework, or in your case, the software), then a book may be one worthwhile component of a monetization strategy. If you can sell a $30 book to 4000 people, you can earn some decent money. (The royalties on a $30 book on Leanpub are $26.50, so multiplying by 4000 results in over $100K.)
It appears this talk was given in the context of the Benjamin Pierce Centennial Conference:
May you have fair winds and following seas.
Congratulations!
Also I'd like to wish you luck with your endeavors!
Back in 2011 or so, as a young undergrad, I latched onto sage and used it for an undergrad research project. My BS was from a tiny university (one year, I was the only Physics major in the school)..and I tried to turn all my friends and profs onto sage, being small meant there was no dept. standard, so I tried to impress it on the dept. (3 people really) but they stuck with mathematica because sage didn't even have an easy to use ode solver! For pete's sake... I understand that sage is a niche project for the math community, but if that's the case, that's the only place you'll find funding and devs from.
This is often said here amongst the startup nerds: make sure you have an audience willing to pay. Hey, many of us in the "more applied community" would love to have a FOSS tool that rivals mathematica, we exist! But it needs to do things well, or at least well enough that in linear combination with the fact that it is open source, the overall goodness vector for the project's value has a timelike norm. Then, we'd clamor for it, you get downloads, and one day, the funders will go, "hey, that's good shit right there, I better be a part of it!"
They don't need to do things for others, or for others' interest. But then, no one should be surprised when such efforts don't get funding. I mean, doing something niche implies that less people will be interested which implies that less people will fund it, right? It's almost a direct consequence of choosing to serve a niche.
Perhaps I can help in making that happen. And so might other people reading this thread!
among other things, SageMath reconciled the confusing namespace soup that is scientific python (numpy, scipy, and matplotlib--three brilliant libraries with partial overlap in functionality and in package names) which SageMath gathered (along with other libraries like SymPy) and put them under a single rubric, 'SageMath'--one (large) install and you have all of scientific python.
SageMath also included a notebook
seems not such a big deal now with Anaconda and Jupyter notebook, but in 2007 it certainly was.
I mean, yes. But, also no.
Prof. Stein, You do not know me, but you have been an inpiration to me. I came across several of your books during a year of post-bac study. They spoke to me, especially "Algebraic Number Theory, a Computational Approach". They also steered me toward your home page, and your work on Sage Math. I thought to myself, "yes. yes!"
Though I'm not punk as fuck, I'm definitely a 'walk to your own drum beat' believer, and a skateboarding professor that heads an open source project taking on Mathematica would make an awesome lodestar. I was 32 when I quit a great job at a very well know Wall Street investment firm (back-office, not master-of-the-universe stuff, but definitely a good place to be) so that I could study nothing but math for a year. I should point out, my math grades up to this point were:
- D in my senior year in high school
- C in the only undergrad math course I had to take
So, everyone was like, "You're effin crazy, what the eff are you doing, you're making an effin bad decision..." Etc. Well, it was the best decision I ever made. Two weeks after leaving my job I was in a dorm room with an 18 year old football player (very, very awkward), but a year later I was a class or two away from a degree in math. My wife and I decided to add moving (again), wedding planning, and another thing to our life, so I didn't quite finish a degree. I received a bunch of As and a few Bs. It was a miracle. (No, it was a lot of hard work, and having seen the light which is the beauty of mathematics).
I've thought many, many, many long hours about the issues of open source development and how it might be made sustainable. I've had to, as it relates intimately to the reason I left my job and went off on this new path. I've got a couple ideas that I believe are very realistically workable. In short, the first go I'll be making at one of these ideas is, software is developed by a community which then makes the source code open source but not compiled into programs, and with no beautify logos or easy to use UIs. They then copy right that code for a month and charge non-members a small (think Spotify) amount to have access to the compiled, bundled, UI'ified versions that are encrypted with a monthly key. Then, at the end of the month, that software is all marked as "old", put in the public domain, and the keys are "unlocked". If the software is useful, the price is right, and the user is not a programmer, then they'll hopefully pay $10 or $15 a month even though they could use last months software for free. Also like Spotify, paying this fee would gain a user access to all the communities software. The subscription fees will be allocated to programmers who will be paid to work on software per rata according to some weighted combo of votes from users and votes by community members. Community members are, of course, free to work on whatever they'd like to in addition to that. Community members receive a payout from the subscription, basically whatever subscription revenue there is minus that paid out for paid development (per previous mentioned mechanism) minus operating expenses. You can only have your software in that "repository" if you are a member, and you must buy in to be a member, sorta like a co-op.
So, that was a very sloppy explanation, but hopefully you get the general points. My main point however, is, please don't go corporate. Even companies like Patagonia, though it is a "B-Corportation" for the public benefit, are clearly driven by the bottom line. How else could one explain why they charge $35 for 40 different types of hats. We don't need 40 different types of hats. But, it drives their bottom line, so that's what we get (albeit, in addition to the great things they also do).
"You know what I hate about f*cking banking? It reduces people to numbers..." You know, the line from "The Big Short". It's not just banking. It's any venture that is driven by the profit motive. Pure and simple.
Profit motive => Reduce everything to numbers
Not right away. Not in a loud and crash fashion. Not one person. But systematically, insidiously, creeping, all together, a step at a time, with the flash of amazing marketing departments and the financial soundness of a well disciplined finance department. Whatever it is you think you're doing, will be metamorphosed into the fungible unit of exchange, like something out of a Kafka book, both absurd and meaningless, while at the same time horrifying.
This 'comment' is very sloppily written because it is being written with some urgency because, (god bless HN, where else will I get to randomly interact with Prof. Stein?) I sincerely believe that you have changed the center of gravity in the world and this is an impassioned plea to keep on keeping on when it comes to helping us that are building a future where (given that intellectual property will clearly make up the bulk of our wealth) the wealth is a well tended commons and not a well guarded garden.
(Speaking briefly to the "academic" angle of things. I understand a bit what the atmosphere is like. I'm going through a divorce at the moment, and my wife just successfully passed her major comps exam and is on her way to a Ph.D. from Johns Hopkins in their political theory department. I am very happy for her and wish her the best, but my point is that, I get the pressures in academia to prioritize certain things while other things, which should really be valued and promoted, are totally overlooked or even punished. But, the business world is not the answer.)
I think this would be a waste of academics. Commercial software is not expensive (mostly) because of some secret sauce. It's because delivering a functioning product requires lots of work that is thoroughly mundane and repeatable.
Analogously, one could employ chemists to bottle coca cola or metallurgists to package hammers but that would be just a waste of everyones assets.
Should government make it's own pencils? I don't think so.
Good products require lots of work that is hard to be intrinsically motivated of.
Last I heard, they switched back to MATLAB. Having taught MATLAB, I wouldn't wish that on anyone. But if SageMathCloud had been around, it would have been a good option.
I may not have time to do a lot but i am gonna join in and help as much as possible. Documentation, bug fixes, whatever. This project deserves it imo
I have some thoughts regarding the comment on Slide15 (making Sage good for some applications): I see that python and Jupyter are very popular for machine learning and allied computations. Can SAGE leverage this to provide a service that a large audience would happily pay for -- and then use that to bootstrap a full fledged mathematical software? (Also, on that note, is there any coherence between the leaders of Sage and Jupyter?)
Basic interface with no requirement to log in, mentioned elsewhere in this discussion. Probably best for 'calculator' mode tryouts.
Academia totally sucks in this deeply ingrained ivory-tower mindset that metes out rewards/tenure/grants based on outdated performance metrics.
(Excuse the vitriol, former academic here.)
The point is that, it should not cost significant continual effort to package SageMath for Debian: if SageMath was following good engineering practises, then Tim Abbott's work would still function today, even taking into account necessarily but normal and minimal maintenance costs that Debian volunteers (including myself) would be happy to do for Sage.
It would be great if you do pay someone to work on this, but please also keep in mind my points about continual costs. To reduce these, Sage upstream (you) does have to change some of its practises.
I have another perspective though; I knew Allan Steel (Magma guy) as an undergrad at Sydney University. He is an extraordinarily smart person and humble and genial as well.
Everyone should be thankful to him and the University of Sydney for having the wisdom to fund the development of Magma.
He's looking for ways to make safe and open source software dominate. And there needs to be a lot of growth for that to happen.
TL;DR: Researcher A finds things he wants improved in Magma (closed source) but can't. Researcher B tries to write improved FOSS implementation, but lost his job, likely in part by spending too much time writing said code and not doing other things like writing papers. Researcher A moves on and has a successful academic career. Moral: writing FOSS can cost you your academic job; it's safer to find something else to do.
sage: lisp('(+ 2 3)')
5
[Edit: Added Microsoft/Google.]
I really applaud William's effort to build a company. I wish him all the best and will be willing to provide what help I can.
I wrote an AMS diatribe http://www.ams.org/notices/201202/rtx120200320p.pdf about publishing computational mathematics, making several points, one of which was that
"Algorithm implementations are proprietary. This needs to end. Science is not done behind a curtain. At least, it has not been hidden since Tar- taglia and Cardano fought over solving the cubic."
However, even if the Ma* software were suddenly open sourced it would be obvious that there was a huge problem.
Mathematics rests on several pillars which are currently absent in computational mathematics.
Mathematics rests on proofs. Where are the proofs for computational mathematical software?
Mathematics rests on research papers, books, and references. Where are the explanations of the theory behind the code? Where are the explanations of the design choices, such as which version of an equation was used and why? Computational mathematics needs much more than bare code.
Mathematics rests on courses and students. There may be one course but a whole focused curriculum on computational mathematics needs to exist.
Mathematics rests on funding. Universities, government, and some government organizations, like Oak Ridge, are the primary support.
William and I have had several discussions around our common problem of finding funding. I was the lead on Magnus (Infinite Group Theory) at City College of New York. We struggled for funding all the time. Axiom can't be funded because there is no organization to handle receipts. Funding agency like accountants.
Indeed, accounting is vital to open source funding. I contacted several large organizations asking them to set up an "open source accounting firm" (OSAF) which would accept and administer the grants to open source projects. OSAF would accept the grant, maintain the account, disburse funds for valid receipts, and maintain financial records for inspection. Such an accounting organization is needed if an open source project is going to get government or company funding.
If SageMath could handle the OSAF issue then the various contributing projects used by Sage could apply for grants from companies or government, knowing that there is an organization capable of managing the funds. This has the non-trivial side benefit of making SageMath the primary focus.
Oh, and SageMath could take the "overhead" (more than 50% at most schools) for "paperwork". My provost lived rather well on the grants.
Source: current Stanford PhD student
