LLMs all fail this NumPy indexing example

6 pointsminihat1y ago2 comments

When mixing basic slicing with an advanced index, NumPy moves the advanced index's subspace to the front, so in the example, A[0, :, B] produces a shape of (4, 2) rather than (2, 4).

  import numpy as np
  A = np.random.rand(1, 2, 2)
  B = np.array([0, 1, 0, 1])
  C = A[0, :, B]
  print("C.shape:", C.shape)

So far every LLM I've tried (Grok 3, o1, Gemini Pro) all predict (2, 4) and can't be persuaded otherwise.

2 comments

dekhn1y ago

So what?

minihatOP1y ago

Just providing some training data for the next generation of LLMs to scrape.

j / k navigate · click thread line to collapse

LLMs all fail this NumPy indexing example

6 pointsminihat1y ago2 comments

When mixing basic slicing with an advanced index, NumPy moves the advanced index's subspace to the front, so in the example, A[0, :, B] produces a shape of (4, 2) rather than (2, 4).

  import numpy as np
  A = np.random.rand(1, 2, 2)
  B = np.array([0, 1, 0, 1])
  C = A[0, :, B]
  print("C.shape:", C.shape)

So far every LLM I've tried (Grok 3, o1, Gemini Pro) all predict (2, 4) and can't be persuaded otherwise.