miguelaeh

I have been thinking about this recently. Many projects are focused on running LLMs and SLMs locally. However, is that just for playing around? Or do you actually want to run the inference locally in your companies?

I feel like there could be 2 major advantages: costs at scale and privacy.

1. When talking about the cost, GPT-4o-mini is inexpensive and if we continue in that path, the cost for inference will become negligible soon. Unless your company makes huge use of the model (or uses huge contexts), like those running thousands of autonomous agents, investing in the hardware, does not seem like the best alternative.

2. Privacy. I would say this is more relevant for some industries that work with highly sensitive data. However, I can see how big companies simply engage in private cloud contracts with Azure or other cloud providers. They provide that peace of mind and scalability and at the same time, depending on the contract, some guarantees.

So my big question is, do you know use cases or companies deploying LLMs on their data centers, or looking to do it or is this just for hobbyists?

I was earlier today reviewing some logs from one of my Nginx instances and found the following entry:

``` 401 GET / Expanse, a Palo Alto Networks company, searches across the global IPv4 space multiple times per day to identify customers' presences on the Internet. If you would like to be excluded from our scans, please send IP addresses/domains to: scaninfo@paloaltonetworks.com ```

I think it is just genius. It is a demonstration of effective marketing and knowing where your audience is.

Yesterday the payment card network in Spain was down for several hours. I personally was in the supermarket and it was chaos. People with all the products in their baskets were piling up without being able to pay. The same thing happened in restaurants and basically anywhere you had to pay, including online payments. The event probably caused losses to the majority of Spanish businesses, and everything occurred because of the failure in the system of a single company. I am genuinely interested on hearing the point of people who still don't believe that decentralized blockchain networks are the future of payments.

Hi!

I am currently considering moving a framework I built from Python to Rust to make it faster and take advantage of all the Rust safe features. However, one of my requirements is to still allow users to use Python code, thus, I was thinking about using RustPython for that. I have been doing basic experiment but I would like to ask if anyone has done that before, and the limitations you found on the road. I have read somewhere that RustPython now seems to support pip packages, but I am also not sure about the limitations of it.

Thanks in advance