Add SVE implementation of replace

[hazzlim]

This PR adds an SVE implementation of replace. This algorithm was previously not vectorized using Neon, due to the absence of masked stores in the instruction set. See #4433 for why this is an issue.

Benchmark results ⏲️

Results are speedup values relative to the existing C code as a baseline - higher is better. Benchmark results were obtained running on a Neoverse N2 machine.

	MSVC Speedup	Clang Speedup
r<std::uint8_t>	17.03	7.024
r<std::uint16_t>	10.17	3.767
r<std::uint32_t>	4.592	2.109
r<std::uint64_t>	2.475	1.23

[StephanTLavavej]

Something is wrong with this PR in the MSVC-internal repo again and I need to pull it from the merge batch.

This is breaking the ARM64-native (arm64chk) compiler such that it ICEs. On a D32ps_v6 VM, the error is uninformative:

### CLI output:
Microsoft (R) C/C++ Optimizing Compiler Version 19.52.36529.98 for ARM64 (PREVIEW)
Copyright (C) Microsoft Corporation.  All rights reserved.

test.cpp
c1xx: fatal error C1001: Internal compiler error.
(compiler file 'msc1.cpp', line 1673)
 To work around this problem, try simplifying or changing the program near the locations listed above.
If possible please provide a repro here: https://developercommunity.visualstudio.com 
Please choose the Technical Support command on the Visual C++ 
 Help menu, or open the Technical Support help file for more information
INTERNAL COMPILER ERROR in 'C:\repos\msvc\binaries\arm64chk\bin\arm64\cl.exe'
    Please choose the Technical Support command on the Visual C++
    Help menu, or open the Technical Support help file for more information

@cdacamar helped out tremendously by getting more repro details, and our partners at Arm are now investigating.

[StephanTLavavej]

I just read (in MSVC-PR-757601) that "The ARM64EC ABI does not include the SVE register state (z/p/FFR are not preserved across EC/x64 transitions - armasm64 rejects them with A4266)", so should we be activating this for original recipe ARM64 only and not extra crispy ARM64EC?

[hazzlim]

I just read (in MSVC-PR-757601) that "The ARM64EC ABI does not include the SVE register state (z/p/FFR are not preserved across EC/x64 transitions - armasm64 rejects them with A4266)", so should we be activating this for original recipe ARM64 only and not extra crispy ARM64EC?

I am just consulting with people more in the know regarding ARM64EC than I, but as far as I understand https://learn.microsoft.com/en-us/windows/arm/arm64ec-abi does indeed imply that SVE Z and P registers are forbidden entirely.

So I guess the fix is to remove _M_ARM64EC from our guards

[AlexGuteniev]

So I guess the fix is to remove _M_ARM64EC from our guards

But can't we keep Neon for _M_ARM64EC and guard away only SVE+_M_ARM64EC ?

[StephanTLavavej]

Yes, my understanding is that ARM64EC supports NEON, so we shouldn't mess with that.

I believe that we should (thanks @AlexGuteniev for remembering the tests):

1. Restore the definition of _VECTORIZED_FOR_X64_X86_ARM64 which had been removed by Enable ARM64EC vector algorithms #6084.
2. Change the "dashboard" for _VECTORIZED_REPLACE to be _VECTORIZED_FOR_X64_X86_ARM64.
3. Change _Use_FEAT_SVE() to not exist for ARM64EC, ensuring that all codepaths that attempt to call it are also guarded for native-ARM64 only.
   • I am not sure exactly which features should be unavailable for ARM64EC, but everything higher than SVE is presumably a slam dunk. It would be great if Arm could confirm the precise set.
4. In the tests, replace_is_vectorized should be always true only for ARM64
5. Removal of test_replace in _CALL_ALL_X64_VECTOR_ALGORITHMS_ON_ARM64EC should be reverted

[AlexGuteniev]

I am just consulting with people more in the know regarding ARM64EC than I, but as far as I understand https://learn.microsoft.com/en-us/windows/arm/arm64ec-abi does indeed imply that SVE Z and P registers are forbidden entirely.

Here is another page with explicit table: https://learn.microsoft.com/en-us/cpp/build/arm64ec-windows-abi-conventions?view=msvc-170#register-mapping-for-integer-registers

I'm wondering if the decision not to support SVE at all on ARM64EC is permanent in long term, or it may be lifted in a near future.