Major outages across ChatGPT and API (opens in new tab)

The main thing as a user is that they require different nudges to get the answer you are after out of them, i.e. different ways of asking or prompt eng'n

I think it's running some kind of heuristic on the output before passing it to the user, because slightly different prompts will sometimes succeed.

ChatGPT's system is smart enough to recognize that fantasy crimes are not serious information about committing real crimes or whatever.

I actually had a discussion with Phind itself recently, in which I said that in order to help me, it seems like it would need to ingest my codebase so that it understands what I am talking about. Without knowing my various models, etc, I don't see how it could write anything but the most trivial functions.

It responded that, yes, it would need to ingest my codebase, but it couldn't.

It was fairly articulate and seemed to understand what I was saying.

So, how do people get value out of Phind? I just don't see how it can help with any case where your function takes or returns a non-trivial class as a parameter. And if can't do that, what is the point?

Nit: your link has a trailing "s" which makes it 404 :)

As a whole I think it works well in tandem with ChatGPT to bounce ideas or get alternate perspectives.

(I also love the annotation feature where it shows the websites that it pulled the information from, very well done)

"The inference service may be temporarily unavailable - we have alerts for this and will be fixing it soon."

This entire thing is hallucinated as far as I can tell. The links to docs are nice though

Edit: changing “astrojs” to “vite” responds with a really good and accurate answer: https://www.phind.com/search?cache=rh6s7pydzi3312b7rf43i7cm.

Quite impressed

Is /s a self-fulfilling sarcasm indicator or a typo?

Do they get system updates at the same time as the OpenAI API? Is the pricing the same?

and by the time you're done with the handlebar, the electric bike engine has been upgraded 4 times and is now better than any combustion engine, so why bother replacing it?

By the time you get to the market, the bike's engine has gone through multiple more updates, is fully self-driving and can fly. It is also self-replicating and now your buyers might not need you anymore...

I'm still surprised by the problems with it. Last month it lied about some facts then claimed to have sent an email when asked for more details.[1]

Then apologized for claiming to send an email since it definitely did not and "knew" it could not.

It's like a friend who can't say 'I don't know' and just lies instead.

1. I was asking if the 'Christ the King' statue in Lisbon ever had a market in it, a rumor told to me by a local. It did not, contrary to Bard's belief.

I had a bug that wouldn't let me login to my work OpenAI account at my new job 9 months ago. It took them 6 months to respond to my support request and they gave me a generic copy/paste answer that had nothing to do with my problem. We spend tons and tons of money with them and we could not get anyone to respond or get on a phone. I had to ask my coworkers to generate keys for everything. One day, about 8 months later, it just started working again out of nowhere.

We switched to Azure OpenAI Service right after that because OpenAI's platform is just so atrociously bad for any serious enterprise to work with.

The offline service was still working, and people were doing their job.

The online service was not working, and it was causing other people to be unable to do their job. We had 0 control over the third party.

The other thing, I make software and I basically don't touch it for a few years or ever. These third party services are always updating and breaking causing us to update as well.

IB4 let me write my own compilers so I have real control.

Wikipedia has a nice example of an oil stain vs. oil spill. https://en.wikipedia.org/wiki/False_equivalence

But not the features themselves, not so much.

It saved a bunch of manual work on a throwaway script. In the past, I might have done something in Python, since I'm more familiar with it than powershell. Or, I'd say, "well, it's only 20 files. I'll just do it manually." The GPT script worked on the first try, and I just threw it away at the end.

Basically, we use AI to do a lot of formatting for our manuals. It's most useful with the backend XML markups, not WYSIWYG editors.

So, we take the inputs from engineers and other stakeholders, essentially in email formats. Then we pass it through prompts that we've been working on for a while. Then it'll output working XML that we can use with a tad bit of clean-up (though that's been decreasing).

It's a lot more complicated than just that, of course, but that's the basics.

Also, it's been really nice to see these chat based AIs helping others code. Some of the manuals team is essentially illiterate when it comes to code. This time last year, they were at best able to use excel. Now, with the AIs, they're writing Python code of moderate complexity to do tasks for themselves and the team. None of it is by any means 'good' coding, it's total hacks. But it's really nice to see them come up to speed and get things done. To see the magic of coding manifest itself in, for example, 50 year old copy editors that never thought they were smart enough. The hand-holding nature of these AIs is just what they needed to make the jump.

Here's a session from me working on a side project yesterday:

https://chat.openai.com/share/a6928c16-1c18-4c08-ae02-82538d...

The most impressive thing I think starts in the middle:

* I paste in some SQL tables and the golang structrues I wanted stuff to go into, and described in words what I wanted; and it generated a multi-level query with several joins, and then some post-processing in golang to put it into the form I'd asked for.

* I say, "if you do X, you can use slices instead of a map", and it rewrites the post-processing to use slices instead of a map

* I say, "Can you rewrite the query in goqu, using these constants?" and it does.

I didn't take a record of it, but a few months ago I was doing some data analysis, and I pasted in a quite complex SQL query I'd written a year earlier (the last time I was doing this analysis), and said "Can you modify it to group all rows less than 1% of the total into a single row labelled 'Other'?" And the resulting query worked out of the box.

It's basically like having a coding minion.

Once there's a better interface for accessing and modifying your local files / buffers, I'm sure it will become even more useful.

EDIT: Oh, and Monday I asked, "This query is super slow; can you think of a way to make it faster?" And it said, "Query looks fine; do you have indexes on X Y and Z columns of the various tables?" I said, "No; can you write me SQL to add those indexes?" Then ran the SQL to create indexes, and the query went from taking >10 minutes to taking 2 seconds.

(As you can tell, I'm neither a web dev nor a database dev...)

I also use it heavily for formatting adjustments. Instead of hand-formatting a transcript I pull from YouTube, I paste it into Claude and have it reformat the transcript into something more like paragraphs. Many otherwise tedious reformatting tasks can be simplified with an LLM.

I also will get an LLM to develop flashcards for a given set of notes to drill on, which is nice, though I usually have to heavily edit the output to include everything I think I should study.

In class, if I'm falling behind on notetaking, I'll get the LLM to generate the note I'm trying to write down by just asking it a basic question, like: "What is anarchism in a sentence?" That way I can focus on what the teacher is saying while the LLM keeps my notes relevant. I'll skim what it generates and edit to fit what my prof said, but it's nice because I can pay better attention than if I feel I have to keep track of what the prof might test me on. This actually is a note-taking technique I've learned about where you only write down the question and look up the answer later, but I think it's nice I now can do the lookup right there and tailor it to exactly how the prof is phrasing it/what they're focusing on about the topic.

What I'm about to say is in the context of programming. I have the tendency to get caught up in some trivial functionality, thus losing focus on the overall larger and greater objective.

If I need to create some trivial functionality, I start with unit tests and a stubbed out function (defining the shape of the input). I enumerate sufficient input/output test cases to provide context for what I want the function to do.

Then I ask copilot/ChatGPT to define the function's implementation. It sometimes takes time to tune the dialog or add some edge cases to the the test cases, but more often than not copilot comes through.

Then I'm back to focusing on the original objective. This has been a game changer for me.

(Of course you should be careful about what code is generated and what it's ultimately doing.)

It's a bit different from other plugins which only act on the text in the buffer in that it also sends the diagnostics from the LSP to ChatGPT too.

https://github.com/piersolenski/wtf.nvim

I wrote a couple commandline tools to do things like autogenerate commit comments or ask it questions from the commandline and return the right bash invocation to do whatever I need done https://github.com/pmarreck/dotfiles/blob/master/bin/functio...

Random thing I did this morning was see if it could come up with an inspiring speech to start healing the rift between israel and its neighbors https://chat.openai.com/share/71498f5f-3672-47cd-ad9a-154c3f...

It's very good at returning unbiased language

Ask it to document the conditions according to the code and taking into consideration the following x, y, z.

Output a raw markdown table with the columns a, b, c.

Translate column a in English between ()

---

Speeds up the "document what you're doing" for management purpose, while I'm actually coding and testing out scenarios.

Tbh. I'm probably one of the few that did the coding while "doing the analysis".

Ps. It's also great for writing unit tests according to arrange, act, assert.

In the end I settled on a standalone desktop app to "compose" prompt with source code, instructions and formatting options which I can just copy paste into ChatGPT.

The app is available for download if anyone is interested: https://prompt.16x.engineer/

Using it for basically every component of my startup.

Image generation and image interpretation means I may never hire a designer.

I mostly use it for writing and debugging small Bash and Python scripts, and creating tables and figures in LaTeX.

For coding I’ve been running https://huggingface.co/TheBloke/Phind-CodeLlama-34B-v2-GGUF locally for the past couple of days and it’s impressive. I’m just using it for a small web app side project but so far it’s given me plenty of fully functional code examples, explanations, help with setup and testing, and occasional sass (I complained that minimist was big for a command line parser and it told me to use process.env ‘as per the above examples’ if I wanted something smaller.)

Time to see how unreliable OpenAI's API is just like when GitHub has an outage every week, guaranteed.

[0] https://news.ycombinator.com/item?id=36063608

For general cloud, avoiding screwing might mean multi cloud. But for LLM, there’s only one option at the highest level of quality for now.

People tend to over focus on resilience (minimizing probability of breaking) and neglect the plan for recovery when things do break.

I can’t tell you how weirdly foreign this is to many people, how many meetings I’ve been in where I ask what the plan is when it fails, and someone starts explaining RAID6 or BGP or something, with no actual plan, other than “it’s really unlikely to fail”, which old dogs know isn’t true.

I guess the point is, for now, we’re all de facto plug-in authors.

As more models are released, it becomes possible to integrate directly in some stacks (such as Elixir) without "direct" third-party reliance (except you still depend on a model, of course).

For instance, see:

- https://www.youtube.com/watch?v=HK38-HIK6NA (in "LiveBook", but the same code would go inside an app, in a way that is quite easy to adapt)

- https://news.livebook.dev/speech-to-text-with-whisper-timest... for the companion blog post

I have already seen more than a few people running SaaS app on twitter complaining about AI-downtime :-)

Of course, it will also come with a (maintenance) cost (but like external dependencies), as I described here:

https://twitter.com/thibaut_barrere/status/17221729157334307...

We might see SETI-like distributed training networks and specific permutations of open source licensing (for code and content) intended to address dystopian AI scenarios.

It's only been a few years since we as a society learned that LLMs can be useful in this way, and OpenAI is managing to stay in the lead for now, though one could see in his facial countenance that Satya wants to fully own it so I think we can expect a MS acquisition to close within the next year and will be the most Microsoft has ever paid to acquire a company.

MS could justify tremendous capital expenditure to get a clear lead over Google both in terms of product and IP related concerns.

Also, from the standpoint of LLMs, Microsoft has far, far more proprietary data that would be valuable for training than any other company in the world.

Gonna be similar (or worse) to what happens when Github goes down. It amazes me how quickly people have come to rely on "AI" to do their work for them.

But...are we? There's a reason that many enterprises that need reliability aren't doing that, but instead...

> It took years before most companies who now use cloud providers to trust and be willing to bet their operations on them. That gave the cloud providers time to make their systems more robust, and to learn how to resolve issues quickly.

...to the extent that they are building dependencies on hosted AI services, doing it with traditional cloud providers hosted solutions, not first party hosting by AI development firms that aren't general enterprise cloud providers (e.g., for OpenAI models, using Azure OpenAI rather than OpenAI directly, for a bunch of others, AWS Bedrock.)

Right now everyone is scrambling to just get some basic products out using LLMs but as people have more breathing room I can't image most teams not having a non-OpenAI LLM that they are using to run experiments on.

At the end of the day, OpenAI is just an API, so it's not an incredibly difficult piece of infrastructure to have a back up for.

Some of the tips in this discussion threads are invaluable and feel good for where I might already be thinking about some things and other new things to think about.

Commenting separately on those below.

You said it so well!

Cloud =! OpenAI

Clouds store and process shareable information that multiple participants can access. Otherwise AI agents == new applications. OpenAI is the wrong evolution for the future of AI agents

https://invidious.flokinet.to/watch?v=DP4-Zp-fBXc

That's so cool. And horrifying. It's like back when Twitter was one global feed on the front page. I doubt that's intended behavior since this URL is generated by the share link.

Be forewarned.

--

Here you go: https://www.phind.com/search?cache=nsa0xrak9gzn6yxwczxnqsck

- Can't work, no computers.

- Can't work, no internet.

- Can't work, no Google.

- Can't work, no ChatGPT.

- Can't work, no xxxxxx?

I've had it generate some regexes and answer questions when I can't think of good keywords; but half of my searches are things where I'm just trying to get to the original docs; or where I want to see a discussion on an error message.

Maybe @sama can help you (or anyone else that has a ChatGPT wrapper app) :P

People get credits for 'outages', but if it is sometimes working for someone somewhere then that is the convenient fiction/loophole a lot of companies use.

Green Check with (i) notice

https://youtu.be/-tVuxZCwXos?si=4AJ6rPMEa8hPyaaG