undefined | Better HN

0 pointsshagie3y ago0 comments

The thing is that it is good at classifying things.

You could then use it to classify things that it is bad at and use that information (as part of a larger whole) to dispatch a query to the knowledge system that can return back the proper (current) information, and report on that.

    The following is a list of questions.  Identify the category they belong to as one of {Current Events}, {General knowlege}, {Unit conversion}, {Math}:

    1. How many feet in a mile?
    2. What is the square root of 541696?
    3. Who is the Speaker of the House?
    4. How many turkeys in Turkey?

to which it responds:

    1. Unit conversion
    2. Math
    3. Current Events
    4. General Knowledge

The supervisor system (for lack of a better word) would then dispatch the questions to different systems that it is coded to be able to either further classify the question or provide the proper answer.

0 comments

pdonis3y ago

> The thing is that it is good at classifying things.

If by "classifying things" you mean doing what Wolfram Alpha's natural language front end does--figuring out which solution engine to pass the question to, and translating the question into what that solution engine needs--then yes. That's what your examples show. And that concedes my point: that something like GPT-3 might make a good natural language front end for something like Wolfram Alpha's solution engines; but it's not good at actually giving correct answers on its own. It's bad at that, for every type of question.

shagieOP3y ago

Yep - the things I've found it good at is stuff like "here is data, answer questions about it" (I've got dabblingings in writing a private Alexa skill that does a better weather report than the rules based one that is currently used) or zero shot classifications. For example with the zero shot: https://news.ycombinator.com/item?id=34156626

And this is a "not everything goes to Wolfram" - the weather report that it has ( https://www.wolframalpha.com/input?i=chicago+weather ) doesn't provide quite the right data (though it is rather neat).

The other thing I need to look at is the question are some of the other, simpler models acceptable for these classifications. The Curie model is 10x cheaper than the Davinci model. And Babbage and Ada are even cheaper.

The system is interesting... but even with some dabbling one can see that this isn't something to trust for knowledge, but rather work with for translating one "language" into another and then using that in conjunction with another layer that can use that translated "language".

j / k navigate · click thread line to collapse