Many test failures with System.IO.InvalidDataException : End of Central Directory record could not be found.

[tarekgh]

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=1512421
Build error leg or test failing: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowGettingSchemaMultipleTimes
Pull request: https://github.com/dotnet/machinelearning.git/pull/7661

System.IO.InvalidDataException : End of Central Directory record could not be found

   at System.IO.Compression.ZipArchive.ReadEndOfCentralDirectory()
   at System.IO.Compression.ZipArchive..ctor(Stream stream, ZipArchiveMode mode, Boolean leaveOpen, Encoding entryNameEncoding)
   at System.IO.Compression.ZipFile.Open(String archiveFileName, ZipArchiveMode mode, Encoding entryNameEncoding)
   at System.IO.Compression.ZipFile.ExtractToDirectory(String sourceArchiveFileName, String destinationDirectoryName, Encoding entryNameEncoding, Boolean overwriteFiles)
   at Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.UnZip(String gzArchiveName, String destFolder) in D:\a\_work\1\s\test\Microsoft.ML.TensorFlow.Tests\TensorflowTests.cs:line 2015
   at Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.DownloadImageSet(String imagesDownloadFolder) in D:\a\_work\1\s\test\Microsoft.ML.TensorFlow.Tests\TensorflowTests.cs:line 1958
   at Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests..ctor(ITestOutputHelper output) in D:\a\_work\1\s\test\Microsoft.ML.TensorFlow.Tests\TensorflowTests.cs:line 73
   at InvokeStub_TensorFlowScenariosTests..ctor(Object, Span`1)
   at System.Reflection.MethodBaseInvoker.InvokeWithOneArg(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

DO NOT USE JSON BELOW IF THIS IS A BUILD BREAK otherwise build analysis will allow pull requests to merge that break the build worse. For a build break, do not use this issue form. Make a regular new issue.

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "System.IO.InvalidDataException : End of Central Directory record could not be found",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=1512421
Error message validated: [System.IO.InvalidDataException : End of Central Directory record could not be found]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 7/16/2026 3:25:44 PM UTC

Report

Build	Repository	Test	Pull Request
1513035	dotnet/machinelearning	Microsoft.ML.AutoML.Test.AutoFitTests.AutoFitMultiClassification_Image_CV
1513034	dotnet/machinelearning	Microsoft.ML.AutoML.Test.AutoFeaturizerTests.AutoFeaturizer_image_test
1512779	dotnet/machinelearning	Microsoft.ML.AutoML.Test.AutoFitTests.AutoFitMultiClassification_Image_CV
1512778	dotnet/machinelearning	Microsoft.ML.AutoML.Test.AutoFeaturizerTests.AutoFeaturizer_image_test
1512421	dotnet/machinelearning	Microsoft.ML.AutoML.Test.AutoFeaturizerTests.AutoFeaturizer_image_test	#7661

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
5	5	5

[github-actions]

🔍 No similar issues found with high confidence.

---

AI-powered similar issue detection

[ericstj]

I think this is a download failure for the tests fetching a data set. https://aka.ms/mlnet-resources/datasets/flower_photos_tiny_set_for_unit_test.zip is failing. Looking at https://witty-sand-06796de1e.2.azurestaticapps.net/links?host=1&q=mlnet-resources I don't see a registration for this. I haven't been able to find the actual path but it should be on https://mlpublicassets.blob.core.windows.net.

[ericstj]

Yep, confirmed those aka.ms links are gone. We need a directory listing of https://mlpublicassets.blob.core.windows.net/ to restore them.

[ericstj]

ML.NET aka.ms/mlnet-resources audit

Audited 2026-07-16 for dotnet/machinelearning issue #7662.

Summary

• 37 logical resources are referenced by the current repository.
• 39 exact URL forms are generated because:
  • datasets/flower_photos_small_set.zip also appears with a trailing /.
  • Text/Sswe/sentiment.emd also appears as lowercase text/Sswe/sentiment.emd.
• 24 logical resources still resolve to the expected asset.
• 13 logical resources redirect to Bing HTML instead of an asset.
• Eight of the broken aliases have clear matching blobs in the public assets container.
• Five broken benchmark aliases have no matching blob anywhere in the 33-object public container. Their expected files are checked into test/data.

Probe method: streamed GET with Range: bytes=0-63, following redirects. An alias was considered healthy only when its final response was non-HTML and its file signature/content type matched the expected asset.

Broken aliases with recoverable blob targets

Alias path under https://aka.ms/mlnet-resources/	Expected target	Size	Evidence
datasets/flower_photos_small_set.zip	https://mlpublicassets.blob.core.windows.net/assets/datasets/flower_photos_small_set.zip	11,709,383	ZIP signature PK; blob listing MD5 brq+jovspdD1r7EUYoyEVg==
datasets/flower_photos_small_set.zip/	Same target as above; trailing-slash variant used by the performance benchmark	11,709,383	ZIP signature PK
datasets/flower_photos_tiny_set_for_unit_test.zip	https://mlpublicassets.blob.core.windows.net/assets/datasets/flower_photos_tiny_set_for_unit_tests.zip	2,324,722	Historical PR #4523 confirms singular alias to plural filename; ZIP signature PK; blob MD5 OBxMEK1IO9zlxzMZ5fZeYw==
datasets/CatsVsDogs_tiny_for_unit_tests.zip	https://mlpublicassets.blob.core.windows.net/assets/datasets/CatsVsDogs_tiny_for_unit_tests.zip	3,106,836	Historical PR #4523; ZIP signature PK; blob MD5 DLV9zSWlPVX18Cy9LtbRuQ==
benchmarks/external/MSLRWeb10KTrain720kRows.tsv	https://mlpublicassets.blob.core.windows.net/assets/benchmarks/MSLRWeb10KTrain720kRows.tsv	519,712,566	TSV content; blob MD5 L/9o53RKnfL94+E9OskucA==
benchmarks/external/MSLRWeb10KValidate240kRows.tsv	https://mlpublicassets.blob.core.windows.net/assets/benchmarks/MSLRWeb10KValidate240kRows.tsv	169,246,139	TSV content; blob MD5 Vc8t62qTYTOVNRU7sNqbNw==
benchmarks/external/MSLRWeb10KTest240kRows.tsv	https://mlpublicassets.blob.core.windows.net/assets/benchmarks/MSLRWeb10KTest240kRows.tsv	172,985,414	TSV content; blob MD5 L47StrmjXdvrWEEhd84tkA==
benchmarks/external/digits.csv	https://mlpublicassets.blob.core.windows.net/assets/benchmarks/digits.csv	264,712	CSV content; blob MD5 oDOcMKilMSobb55ccZ3M5Q==
benchmarks/external/WikiDetoxAnnotated160kRows.tsv	https://mlpublicassets.blob.core.windows.net/assets/benchmarks/wikiDetoxAnnotated160kRows.tsv	78,507,299	Alias uses Wiki...; blob uses wiki...; blob MD5 gq/IWOA/spmks/+h1um+Fg==

Every alias above currently redirects to https://www.bing.com/?ref=aka&shorturl=... and returns HTML.

Broken aliases without a matching public blob

These files exist in test/data and are normally found locally before the fallback download is attempted. The public assets container contains no object with a matching basename.

Alias path	Existing asset identity	Raw main candidate	Raw size
benchmarks/backcompat/MurmurHashV1.zip	test/data/backcompat/MurmurHashV1.zip	https://raw.githubusercontent.com/dotnet/machinelearning/main/test/data/backcompat/MurmurHashV1.zip	791
benchmarks/adult.with-schema.txt	test/data/adult.with-schema.txt	https://raw.githubusercontent.com/dotnet/machinelearning/main/test/data/adult.with-schema.txt	3,656,858
benchmarks/iris.txt	test/data/iris.txt	https://raw.githubusercontent.com/dotnet/machinelearning/main/test/data/iris.txt	2,757
benchmarks/wikipedia-detox-250-line-data.tsv	test/data/wikipedia-detox-250-line-data.tsv	https://raw.githubusercontent.com/dotnet/machinelearning/main/test/data/wikipedia-detox-250-line-data.tsv	72,136
benchmarks/breast-cancer.txt	test/data/breast-cancer.txt	https://raw.githubusercontent.com/dotnet/machinelearning/main/test/data/breast-cancer.txt	15,173

For durable restoration, uploading immutable copies to mlpublicassets is safer than targeting the moving main branch.

Healthy aliases

Alias path	Current final target	Size
image/AlexNetPrepOnnx/AlexNetPreprocess.onnx	assets/image/AlexNetPrepOnnx/AlexNetPreprocess.onnx	618,820
image/ResNetPrepOnnx/ResNetPreprocess.onnx	assets/image/ResNetPrepOnnx/ResNetPreprocess.onnx	602,584
image/AlexNetOnnx/AlexNet.onnx	assets/image/AlexNetOnnx/AlexNet.onnx	233,133,636
image/ResNet18Onnx/ResNet18.onnx	assets/image/ResNet18Onnx/ResNet18.onnx	63,653,533
image/ResNet50Onnx/ResNet50.onnx	assets/image/ResNet50Onnx/ResNet50.onnx	94,288,003
image/ResNet101Onnx/ResNet101.onnx	assets/image/ResNet101Onnx/ResNet101.onnx	170,508,881
WordVectors/glove.6B.50d.txt	assets/wordvectors/glove.6B.50d.txt	171,350,079
WordVectors/glove.6B.100d.txt	assets/wordvectors/glove.6B.100d.txt	347,116,733
WordVectors/glove.6B.200d.txt	assets/wordvectors/glove.6B.200d.txt	693,432,828
WordVectors/glove.6B.300d.txt	assets/wordvectors/glove.6B.300d.txt	1,037,962,819
WordVectors/glove.twitter.27B.25d.txt	assets/wordvectors/glove.twitter.27B.25d.txt	257,699,726
WordVectors/glove.twitter.27B.50d.txt	assets/wordvectors/glove.twitter.27B.50d.txt	510,887,943
WordVectors/glove.twitter.27B.100d.txt	assets/wordvectors/glove.twitter.27B.100d.txt	1,021,669,379
WordVectors/glove.twitter.27B.200d.txt	assets/wordvectors/glove.twitter.27B.200d.txt	2,057,590,469
WordVectors/wiki.en.vec	assets/wordvectors/wiki.en.vec	6,597,238,061
Text/Sswe/sentiment.emd	assets/Text/Sswe/sentiment.emd	73,674,434
text/Sswe/sentiment.emd	Same target as above	73,674,434
meta/resnet_v2_101_299.meta	assets/meta/resnet_v2_101_299.meta	179,017,396
meta/inception_v3.meta	assets/meta/inception_v3.meta	95,664,402
meta/mobilenet_v2.meta	assets/meta/mobilenet_v2.meta	24,532,166
meta/resnet_v2_50_299.meta	assets/meta/resnet_v2_50_299.meta	102,616,931
models/NasBert2000000.tsm	assets/torchsharp/text/encoder_2000000.tsm	17,907,563
models/pretrained_NasBert_14M_encoder.tsm	assets/torchsharp/text/pretrained_NasBert_14M_encoder.tsm	54,767,471
models/pretrained_Roberta_encoder.tsm	assets/torchsharp/text/pretrained_Roberta_encoder.tsm	496,226,290
models/autoformer_11m_torchsharp.bin	assets/torchsharp/obj-detection/autoformer_11m_torchsharp.bin	77,547,537

All healthy aliases return a 301 redirect to https://mlpublicassets.blob.core.windows.net/...; the target returned a valid partial-content response and the expected file signature.

Repository source locations

• Base URL and dynamic construction: src/Microsoft.ML.Core/Utilities/ResourceManagerUtils.cs:36,84,108
• Image ONNX aliases: src/Microsoft.ML.DnnImageFeaturizer.ModelRedist/Microsoft.ML.DnnImageFeaturizer.ModelRedist.csproj:9-35
• Flower dataset aliases:
  • docs/samples/Microsoft.ML.Samples/Dynamic/Trainers/MulticlassClassification/ImageClassification/ImageClassificationDefault.cs:245-246
  • docs/samples/Microsoft.ML.Samples/Dynamic/Trainers/MulticlassClassification/ImageClassification/ResnetV2101TransferLearningEarlyStopping.cs:233-234
  • docs/samples/Microsoft.ML.Samples/Dynamic/Trainers/MulticlassClassification/ImageClassification/ResnetV2101TransferLearningTrainTestSplit.cs:254-255
  • test/Microsoft.ML.AutoML.Tests/DatasetUtil.cs:187-188
  • test/Microsoft.ML.PerformanceTests/ImageClassificationBench.cs:141-142
  • test/Microsoft.ML.TensorFlow.Tests/TensorflowTests.cs:1953-1968
• Word embeddings: src/Microsoft.ML.Transforms/Text/WordEmbeddingsExtractor.cs:605-634
• TensorFlow meta-graphs: src/Microsoft.ML.Vision/ImageClassificationTrainer.cs:101-106,1312-1315
• TorchSharp model aliases:
  • src/Microsoft.ML.TorchSharp/NasBert/TextClassificationTrainer.cs:115
  • src/Microsoft.ML.TorchSharp/NasBert/SentenceSimilarityTrainer.cs:115
  • src/Microsoft.ML.TorchSharp/NasBert/NerTrainer.cs:118
  • src/Microsoft.ML.TorchSharp/Roberta/QATrainer.cs:100
  • src/Microsoft.ML.TorchSharp/AutoFormerV2/ObjectDetectionTrainer.cs:119
• Benchmark URL construction and call sites:
  • test/Microsoft.ML.PerformanceTests/BenchmarkBase.cs:32-56
  • test/Microsoft.ML.PerformanceTests/HashBench.cs:103
  • test/Microsoft.ML.PerformanceTests/KMeansAndLogisticRegressionBench.cs:16
  • test/Microsoft.ML.PerformanceTests/PredictionEngineBench.cs:36,75,110
  • test/Microsoft.ML.PerformanceTests/Numeric/Ranking.cs:25-26,75-77
  • test/Microsoft.ML.PerformanceTests/RffTransform.cs:22
  • test/Microsoft.ML.PerformanceTests/Text/MultiClassClassification.cs:23,100
  • test/Microsoft.ML.PerformanceTests/TextPredictionEngineCreation.cs:25
  • Dataset names: test/Microsoft.ML.TestFrameworkCommon/Datasets.cs:159-166,200-217

Public container inventory

Anonymous listing is enabled:

https://mlpublicassets.blob.core.windows.net/assets?restype=container&comp=list

The complete listing currently contains 33 blobs.