How would you describe what ML can offer me, a non-ML-saavy tech organization?
When does it work better than manually programming something?
Which kinds of data work best?
How extreme of data can the model be fed before giving non-sense results? And will it "know" how hard it is guessing?
What is the format of its results?
... etc ...
Tangentially, I may have many, many more follow-up questions. Are there any freelance "AI Consultants"? And if so, where best to find them?
What can ML offer?
ML allows you to more easily make predictions from data you already have. An example is taking search terms and grouping them into their subject matter, or taking images and identifying things inside them visually. If you deal in sales, you could predict future sales based on other factors that you believe are correlated (weather? clicks?). If you deal with customers, you could more quickly identify what type of issues people are having (hardware issues? card failures?), or proactively suggest solutions.
Better than programming:
It works better when, to code a new feature, you'd need an abundance IF statements whose logic you'd have to work out by hand. In a very broad way, the machine is figuring out how to approach your task just with lots of examples and scary math. You can get a lot of different "answers" without having to write that much more code, just by trying different data and structures.
Practically it's easier than programming to keep updated as well, because if you can find example inputs that it guessed wrong, you can get automatic improvements to results by retraining with correct predictions.
But of course, the ML system itself is very complex, and it involves a lot of resources to design it, curate input data, and train on costly GPUs. This combination of the system's design, and the learned state of the systems innards, is called "the model".
Input data:
There are many different ML systems designed to use different kinds of input. Some use pure text, image data, audio data, graphs of interconnected thingies. The easiest is when the data is very uniform in structure, grouped into labeled columns (called "features"), with each column having a meaningful value. To train the system, you must also supply a "target" feature for each record, which are examples of what you'd like the system to predict. Anything that fits naturally in an Excel sheet might be a starting point. Generally, you want quite a few examples, but the exact amount of data you'd need varies with how difficult your task is.
Bad input, bad results:
Varies. From the practical side - ML systems usually produce confidence scores which you can use to avoid embarrassment. You can then manually label those confusing examples and feed them back in for training. In terms of the model itself - there are many ways of interpreting and evaluating accuracy, and the system can give you examples that confuse it.
Results, two common examples:
Some systems produce a number as their output - called regression. (Picture a sales prediction.)
Others group things into a set of pre-defined categories - called classification. (Picture a system that can tell if there's a giraffe in an image.)
What those outputs mean, and how they are to be used, is part of the design of the model.
Perhaps you can help me with some follow-up questions:
- Let's imagine a standard excel sheet (a 2d array), with columns "A", "B", "C", "D" ... "Z".
1) Let's say I want to create a model, that takes this input: A single row where all columns have values, except for some (random) columns. I want it to autopopulate those columns with values that are most probable according to the data I trained it with. That is, _every_ column is both a "feature" and a "target". Is this possible?
2) Can I train the model by telling it this: "This input should DEFINITELY NOT confuse you" -- that is, can I "weigh" the inputs (or do I just put in more of them?)
2. That seems logical -- supplying a "confidence level" with the training data itself -- but I haven't heard of it, and can't seem to find anything on the search engines.
What if the data I have is relational? Must it all be squashed down? Imagine a family tree: can a "node" simply contain "parents", and the model will figure out "virtual columns", or would I generate this input manually (for thousands of columns), example columns: "parent(0).parent(0).child(0).blood_type", "parent(0).parent(0).child(1).blood_type", etc.
Consequentially... I will definitely have more questions like this. What is a good resource to go off on my own, or with a more suitable audience?
There are graph-based systems but I think they're more interested in understanding relationships in the graph itself - grouping like things together, predicting relationships and distances, etc., rather than attributes of the nodes.
I assume you want the system to learn something like "if parent A is type O, and grandparent of parent B is B+, child is more likely to be tall" or similar. I don't think the network can learn things like that in terms of understanding the linkages to predict numbers. It might be able to predict it simply by being given enough examples after "flattening," though, so the functional result is similar.
I've found Reddit's MLQuestions [0] group to be interesting and sometimes accessible for a non-academic ML enthusiast. Some Youtube content can be useful too, but most are just repeating the content of papers. I'm still seeking a real commoner-level message board to discuss this kind of stuff without dumbshaming.
What I've learned most from is downloading example code and actually using it, reading parts of it, and trying to apply it. It's easy to dip your toes into stuff on Google Colab notebooks.
Uber Ludwig [2] is an interesting all-in-one low-code system that lets you try out different ideas quickly. At least in theory. They give an example cases for all the different types of networks they support along with matching YAML to specify the model details. So you can sorta just throw some data in there and build a command line to try ideas, rather than learning a lot about Pytorch, Keras, etc., and potentially introducing subtle bugs.
Email me (addr in profile) if you want to chat more.
[1] http://reddit.com/r/mlquestions/ [2] https://uber.github.io/ludwig/
