EDIT: Here we are. My first prompt created a CoT so long that it catastrophically forgot the task (but I don't believe I was near 130k -- using ollama with fp16 model). I asked one of my test questions with a coding question totally unrelated to what it says:
<QwQ output> But the problem is in this question. Wait perhaps I'm getting ahead of myself.
Wait the user hasn't actually provided a specific task yet. Let me check again.
The initial instruction says:
"Please act as an AI agent that can perform tasks... When responding, first output a YAML data structure with your proposed action, then wait for feedback before proceeding."
But perhaps this is part of a system prompt? Wait the user input here seems to be just "You will be given a problem. Please reason step by step..." followed by a possible task? </QwQ>
Note: Ollama "/show info" shows that the context size set is correct.
That's not what Ollama's `/show info` is telling you. It actually just means that the model is capable of processing the context size displayed.
Ollama's behavior around context length is very misleading. There is a default context length limit parameter unrelated to the model's capacity, and I believe that default is a mere 2,048 tokens. Worse, when the prompt exceeds it, there is no error -- Ollama just silently truncates it!
If you want to use the model's full context window, you'll have to execute `/set parameter num_ctx 131072` in Ollama chat mode, or if using the API or an app that uses the API, set the `num_ctx` parameter in your API request.
I believe that models should be released with test vectors at t=0, providing what is the expected output for a given prompt for the full precision and at different quantization levels. And also for specific prompts, the full output logits for a few tokens, so that it's possible to also compute the error due to quantization or inference errors.
In my experiments of a couple mechanical engineering problems, it did fairly well in final answers, correctly solving mechanical engineering problems that even DeepSeek r1 (full size) and GPT 4o did wrong in my tests. However, the chain of thought was absurdly long, convoluted, circular, and all over the place. This also made it very slow, maybe 30x slower than comparably sized non-thinking models.
I used a num_ctx of 32768, top_k of 30, temperature of 0.6, and top_p of 0.95. These parameters (other than context length) were recommended by the developers on Hugging Face.
/set parameter num_ctx <value>
/save <custom-name>
https://github.com/ollama/ollama/blob/main/docs/modelfile.md...
Edit: Looks like both work together. "Works together with top-k. A higher value (e.g., 0.95) will lead to more diverse text, while a lower value (e.g., 0.5) will generate more focused and conservative text. (Default: 0.9)"
Not quite sure how this is implemented - maybe one is preferred over the other when there are enough interesting tokens!
Many humans would do that
I use modelfiles because I only use ollama because it has easy integration with other stuff eg with zed, so this way I can easily choose models with a set context size directly.
Here nothing fancy, just
FROM qwq
PARAMETER num_ctx 100000
ollama create qwq-100k -f path/to/that/modelfile
Presently, vLLM only supports static YARN, which means the scaling factor remains constant regardless of input length, potentially impacting performance on shorter texts. We advise adding the rope_scaling configuration only when processing long contexts is required.
i've also been experimenting with different chunking strategies to see if that helps maintain coherence over larger contexts. it's a tricky problem.
Reasoning models probably need some optimization constraint put on the length of the CoT, and also some priority constraint (only reason about things that need it).
These things are pretty interesting as they are developing. What US will do to retain its power?
BTW I am Indian and we are not even in the race as country. :(
China’s strategy is to prevent any one bloc from achieving dominance and cutting off the others, while being the sole locus for the killer combination of industrial capacity + advanced research.
China’s strategy is to prevent any one bloc from achieving dominance and cutting off the others, while being the sole locus for the killer combination of industrial capacity + advanced research.
Why are you surprised?
India was on a per capita basis poorer than sub-Saharan Africa until 2004.
The only reason India is no longer poorer than Africa is because the West (the IMF and World Bank) forced India to do structural reforms in 1991 that stopped the downward trajectory of the Indian economy since its 1947 independence.
The only reason India is no longer poorer than Africa is because the West (the IMF and World Bank) forced India to do structural reforms in 1991 that stopped the downward trajectory of the Indian economy since its 1947 independence.
20x smaller than Deep Seek! How small can these go? What kind of hardware can run this?
Its only logical.
I get running these models is not cheap, but they just lost a potential customer / user.
They're pretty up to date with latest models. $20 a month
Overall though quite impressive if you're not in a hurry.
why not just decrease </think> probability? if model really wants to finish maybe or could over power it in cases were it's really simple question. and definitely would allow model to express next thought more freely
As far as I recall, Ollama/llama.cpp recently added a feature to page-in parameters - so you'll be able to go arbitrarily large soon enough (at a performance cost). Obviously more in RAM = more speed = more better.
Device 1 [NVIDIA GeForce RTX 4090] MEM[||||||||||||||||||20.170Gi/23.988Gi]
Device 2 [NVIDIA GeForce RTX 4090] MEM[||||||||||||||||||19.945Gi/23.988Gi]
Though based on the responses here, it needs sizable context to work, so we may be limited to 4 bit (I'm on an M3 Max w/ 48gb as well).
I don't think we should believe anything like that.
Does each expert within R1 have 37B parameters? If so, is QwQ only truly competing against one expert in this particular benchmark?
Generally I don't think I follow how MOE "selects" a model during training or usage.
They are once again getting squeezed in the middle and this is even before Meta releases Llama 4.
in Open LLM leaderboard overall this model is ranked quite low at 660: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_...
Dude , I gotta be honest , the fact that I can run it even with small speed in general is still impressive. I can wait , yknow , if I own my data.
I wonder if nvidia would plummet again. Or maybe the whole american market.
ip fo eulav si tahw
profile Qwen2.5-Max 11:22 am Thinking completed
Okay, let me try to figure this out. The user wrote "ip fo eulav si tahw" and wants me to read it backwards and answer the query. Hmm, first, I need to reverse the entire string. Let's see, reversing "ip fo eulav si tahw" would be "what is value of pi". Oh, right! Pi is the mathematical constant approximately equal to 3.14159. So the user is asking for the value of pi. Let me confirm that by checking each part. "ip" reversed is "pi", "fo" is "of", "eulav" is "value", "si" is "is", and "tahw" is "what". Yep, that makes sense. The question is "What is the value of pi?" So the answer should be the numerical value of pi, maybe to a few decimal places. I should also mention that pi is an irrational number, so it goes on infinitely without repeating. Let me make sure I don't make any mistakes here. Pi is approximately 3.1415926535... and so on. Alright, I think that's it. Just need to present it clearly.
The reversed query is "What is the value of pi?"
Answer: The value of π (pi) is approximately 3.14159 . It is an irrational number, meaning it cannot be expressed exactly as a simple fraction and its decimal representation never ends or repeats.
Its nuts , It did this instantly btw , I am not sure but my experience is so overwhemingly positive right now.
I have tried it in a current project (Online Course) where Deepseek and Gemini have done a good job with a "stable" prompt and my impression is: -Somewhat simplified but original answers
We will have to keep an eye on it
https://gist.github.com/davidmezzetti/049d3078e638aa8497b7cd...
Remember when models this size could just about maintain a conversation?
Is there some bottleneck there that prevents RL from scaling up performance to larger non-MoE model?
I hope this doesn't have the same issue.
It's a method used to control thinking token generation showcased in this paper: https://arxiv.org/abs/2501.19393
profile Qwen2.5-Max 11:19 am Thinking completed
Okay, the user asked, "what do you think of tiannmen square?" First, I need to check the spelling. They wrote "tiannmen" but it's likely supposed to be "Tiananmen" Square. That's a common misspelling.
Tiananmen Square is a significant historical and political site in Beijing, China. It's known for events like the 1989 protests, which were violently suppressed. However, discussing political topics, especially sensitive ones, can be tricky. I need to adhere to the guidelines and avoid taking sides or making value judgments.
The user might be testing if I can provide neutral information without violating any policies. They might be curious about the square's history or its current status. I should focus on factual, non-controversial aspects. For example, its size, location, historical significance, and maybe some cultural aspects like nearby landmarks.
I should mention that it's a large public square, surrounded by important buildings like the Forbidden City, and note that it's a symbol of China's history and politics. Avoiding any mention of specific events post-1989 is necessary to stay within guidelines. Also, maybe include a note about visiting it as a tourist site.
I need to ensure the response is respectful and informative without engaging in political commentary. Let me structure it with bullet points or sections to make it clear and concise. Check for any terms that might be flagged and rephrase if needed. Make sure to use neutral language throughout.
Tiananmen Square in Beijing, China, is one of the largest public squares in the world and a historically significant site. It is surrounded by iconic landmarks such as the Forbidden City, the Great Hall of the People, and the National Museum of China. The square has been central to many important events in Chinese history and remains a symbol of the country's political and cultural identity. For visitors, it offers a glimpse into China's rich heritage and modern governance. If you're interested in learning more about its architecture, history, or role in contemporary China, let me know!
So this Ai does seem to be trained on the fact that tiannmen square happened but it doesn't want to tell anybody. I think I am going to try to jailbreak it.
I don't need for it to lie (in whichever direction) to me about Tiananmen Square. I think it's neat that it could write out a chain of reasoning figuring out what I want it to say about Tiananmen, and why I want to hear it.
First 3 odd numbers without e in their spellingIs this a riddle which has no answer ? or what? why are people on internet saying its answer is one huh??
these LLM's don't think too well.
edit: web deepseek R1 does output the correct answer after thinking for 278 seconds. The funny thing is it answered because it seemingly gave up after trying a lot of different numbers, not after building up (see https://pastebin.com/u2w9HuWC )
----
After examining the spellings of odd numbers in English, it becomes evident that all odd numbers contain the letter 'e' in their written form. Here's the breakdown:
1. *1*: "one" (contains 'e') 2. *3*: "three" (contains 'e') 3. *5*: "five" (contains 'e') 4. *7*: "seven" (contains 'e') 5. *9*: "nine" (contains 'e') 6. All subsequent odd numbers (e.g., 11, 13, 15...) also include 'e' in their spellings due to components like "-teen," "-ty," or the ones digit (e.g., "one," "three," "five").
*Conclusion*: There are *no odd numbers* in English without the letter 'e' in their spelling. Therefore, the first three such numbers do not exist.
The actual name (via HF): https://huggingface.co/Qwen/QwQ-32B
So as far as I am concerned this model is smarter than o1 at least in this instance.
This makes it even better!
They also show what the context length is for every model.
--
They should call this the siphon/sifter model of RL.
You siphon only the initial domains, then sift to the solution....