- Chunking can interfer with context boundaries
- Content vectors can differ vastly from question vectors, for this you have to use hypothetical embeddings (they generate artificial questions and store them)
- Instead of saving just one embedding per text-chuck you should store various (text chunk, hypothetical embedding questions, meta data)
- RAG will miserably fail with requests like "summarize the whole document"
- to my knowledge, openAI embeddings aren't performing well, use a embedding that is optimized for question answering or information retrieval and supports multi language. SOTA textual embedding models can be found on the MTEB Leaderboard [2]. Also look into instructorEmbeddings
- the LLM used for the Q&A using your context should be fine-tuned for this task. There are several open (source?) LLMs based on openllama and others, that are fine tuned for information retrieval. They hallucinate less and are sticking to the context given.
1 https://github.com/underlines/awesome-marketing-datascience/...
I’ve been working with RAG for months, too, and it’s vanishingly rare to see anything but toy examples in the wild. This is a solid, concise list of where the dragons are.
Any idea where all the RAG practitioners hang out and trade war stories? Is there a forum or Discord or something?
BriefGPT [2] is implementing this and it uses the following prompt at ingestion-time:
"Given the user's question, please generate a response that mimics the exact format in which the relevant information would appear within a document, even if the information does not exist. The response should not offer explanations, context, or commentary, but should emulate the precise structure in which the answer would be found in a hypothetical document. Factuality is not important, the priority is the hypothetical structure of the excerpt. Use made-up facts to emulate the structure. For example, if the user question is "who are the authors?", the response should be something like 'Authors: John Smith, Jane Doe, and Bob Jones' The user's question is:"
1 https://python.langchain.com/docs/modules/chains/additional/...
I want to have a specialist AI that is trained to help me learn how to use the software. 100% what should be happening.
General AI should know how to do stuff too but having an specialist AI implies that the company/group of people/person is making sure to tune the model.
Just an IMO.
You can ask the same questions to ChatGPT and get the same or better answers.
I also know from personal experience with ChatGPT, that you can use it to:
- convert Python/boto3 to any language that has an AWS SDK
- convert CloudFormation to Terraform or the CDK
- write scripts that use the SDK
You will get the occasional hallucination.
ChatGPT: "I'm sorry, but as of my knowledge cut-off in September 2021, there was no service, tool, or product known as Amazon CodeCatalyst offered by Amazon Web Services (AWS). [...]"
Prompt: how to connect a lambda function to an RDS instance and query it safely?
1. Configure your RDS instance for outbound connections to AWS Lambda. This step involves ensuring that your DB instance is in a public subnet on your VPC and that the instance's "PubliclyAccessible" property is set to true.
There’s also the thing where that’s a positive feedback look and we overflow the worlds corpus with garbage.
I've seen answers like this in Stack Overflow before. Technically correct, but basically broken in every other way: cybersecurity's future is gonna be interesting.
Have you ever noticed how the dumbest people tend be be the loudest, most confident, and most eager to share their opinions? Those are the voices that trained ChatGPT.
It’s a hard pass for me, but I fear I’m going to be forced to used it because the dumbest half of humanity will view it as a peer and use it for everything.
To connect a Lambda function to an RDS (Relational Database Service) instance and query it safely, you can follow these general steps:
1. Configure your Lambda function's execution role: Ensure that your Lambda function has the necessary permissions to access the RDS instance. The role should include the `AWSLambdaVPCAccessExecutionRole` and the appropriate RDS permissions.
2. Place your Lambda function and RDS instance in the same VPC: Ensure that both the Lambda function and the RDS instance are in the same Virtual Private Cloud (VPC) to enable their communication. You can configure the VPC settings in the AWS Management Console.
3. Configure the security group settings: Set up the security groups for your Lambda function and RDS instance to allow the necessary inbound and outbound connections. The Lambda function's security group should allow outbound access to the RDS instance's security group on the RDS port (usually port 3306 for MySQL or 5432 for PostgreSQL).
4. Create a subnet group: If your RDS instance is in a private subnet, create a subnet group that includes the appropriate subnets for your Lambda function to access the RDS instance securely.
5. Connect to the RDS instance from the Lambda function: In your Lambda function code, establish a database connection using the appropriate database client library, such as pymysql for MySQL or psycopg2 for PostgreSQL. Use the connection details (endpoint, username, password, etc.) of your RDS instance.
6. Execute queries safely: To query the RDS instance safely, ensure that your Lambda function's code incorporates secure coding practices. Use parameterized queries or prepared statements to prevent SQL injection attacks. Avoid storing sensitive information (such as database credentials) in your Lambda function code and instead use environment variables or AWS Secrets Manager for secure storage and retrieval.
Remember to regularly update and patch your Lambda function, RDS instance, and associated dependencies to maintain security and stay protected against potential vulnerabilities.
It's worth noting that the exact implementation details may vary depending on your specific use case, programming language, and AWS setup. Consulting the AWS documentation and resources related to Lambda, RDS, and VPC networking can provide more detailed and up-to-date instructions for your specific requirements.
I originally thought this was an official Amazon website...their lawyers would probably say the same thing...
I launched “Last Week in AWS” with AWS in the domain name seven years ago. AWS has never made an issue of it, though they obviously have that option.
I also have the option (and ownership) to migrate to “Last Week in the Cloud” and talk about their competitors, so it’s likely everyone is happier this way—but I confess to not kicking the bear hard enough to find out.
I’m sure you know that your name is brought up frequently inside AWS.
These are fun projects!
How does this work?
Glacier is a term that is not directly mentioned in the provided sources.
Prompt: What is a glacier?
A glacier is a large mass of ice that moves slowly over time due to the accumulation of snow, ice, and other forms of frozen precipitation.
Seems like it’s just using a general model?
https://www.phind.com/search?cache=d0b3a85b-17f9-4def-b8d0-b...
They're all so eager to please they will basically never say "that's actually not possible", and invent some plausibly sounding bullshit.
For beginners who are struggling to tell the difference between 1/ how to do something that's possible, 2/ how to do something that SHOULD be possible but just currently isn't, and 3/ how to do something that is RIDICULOUS and shouldn't even ever be possible, ChatGPT is worse than nothing :(
I've tried the using OpenAI w embeddings (iirc), but this was slow, got expensive quickly, and it struggled to answer questions about the text accurately. Curious if there's better standard approaches now.
I think you have to do lots of experiment on this till you find your best information retrieval strategy
Made this up as reason for dedicated vpc: “Better performance: By using dedicated hardware, you have better control over the performance of your instances in the VPC. This can be beneficial for applications that have stringent performance requirements.”
Beyond that it will simply have the output/personality of a person with rote-memory.
Still a language model and not a facts model.
Also, I'd argue that it is very easy to be worse than AWS's own docs. A chatbot that hallucinates inaccurate answers, while sounding plausible and confident is far worse than documentation that is 100% accurate, but lacking in some areas.
I'd rather receive an answer of "I don't know" or "it isn't documented" than an explanation that seems legit, so I spend 15 hours building a solution in accordance to a LLM's response and then find out that its not possible or real the way it was explained by the LLM.
Asking GPT4 is also consistently less of a headache than asking the devops guy and getting a 20min explanation for a simple question.
For example the terseness / symbols of APL, Perl, or event set notation.
LLMs could train and output the shorter symbolic notation, and it could be expanded for human readability by another program at export.
When asked how to create a private HTTP gateway it happily tells you how to do it , but it’s actually impossible.
When the docs are wrong or misleading you'll still get burned, even if the model doesn't hallucinate responses
how do you use wrangler and glue to make athena tables using terraform
it even gets real time indexing from slack of aws deep java library, and from discord of deepset haystack project
How do I connect two VPCs from separate organizations (VPC peering)
How do I only allow authorized applications to access my S3 bucket (BPA, IAM policies, a lot of best practices docs)
With the bonus effect that you could retrain the LLM as often as you like, as new software is released, and it could always be aware of the latest features (and even the bugs and vulnerabilities.)
""" Complete the following code:
// Find all files with a binary pattern in a directory. Return the list of files and the offset of the match in a tuple fn find_in_files(start_dir: &str, pattern: &[u8]) -> vec<(String, usize)> """
No way this won't be abused shortly.
"how do I avoid high NAT gateway bills when an ECS service keeps downloading the same image over and over?"
It offered three replies. The first and third were outright incorrect, the second was (technically) correct:
https://i.imgur.com/la98cxC.png
Also: I'm assuming you haven't actually secured a license to use the AWS logo.
I ran into these issues when building this for my own company's docs, at least.
Well… Thanks a ton for that!
Great idea and setup, but not quite as helpful as I would like yet.
A: Amazon Bedrock is a programming example that uses the AWS SDK for Java 2.x to work with Amazon EC2. It provides examples and code snippets for programming tasks related to Amazon EC2 using the AWS SDK for Java 2.x.
- Make a VPC. - Add an Instance. - Abstract the region and AZ, into vars.
etc... every time I wanted to change the code, I asked the bot to do the refactor, and it did.
Overall, I'm impressed. It wasn't the most complicated thing, but it didn't dive off the deep end.
No, AWS is not designed to take all your money. AWS offers a variety of payment options and cost-saving measures to help you manage your expenses effectively. ...
