You can work around it by creating a cargo config with a wrapper. Eg:

cat ~/.cargo/config

  [target.x86_64-unknown-freebsd]
  linker = "/home/drewg123/bin/cargo-ld"

Where cargo-ld is just a wrapper:

  #!/bin/sh
  exec /usr/bin/ld -L/usr/local/lib $*