Search results

Missing Cargo.lock

Status: Open.

#96 In mlc-ai/tokenizers-cpp;

· xksc opened on Apr 10, 2026

The Encode and EncodeBatch methods methods unnecessarily require std::string instances

Status: Open.

#95 In mlc-ai/tokenizers-cpp;

· gh-andre opened on Mar 27, 2026

Building from the release tag v0.1.1 will fail with errors

Status: Open.

#94 In mlc-ai/tokenizers-cpp;

· gh-andre opened on Mar 10, 2026

linker error when using aarch64 tokenizer_c.a

Status: Open.

#91 In mlc-ai/tokenizers-cpp;

· NaveenMittal0 opened on Dec 12, 2025

Any plans for publishing ubuntu debians, or portable binaries?

Status: Open.

#89 In mlc-ai/tokenizers-cpp;

· blozano-tt opened on Oct 22, 2025

When compiling with rustc 1.90.0 (rustup version 1.28.2), an error was encountered

Status: Open.

#88 In mlc-ai/tokenizers-cpp;

· yangxudong opened on Sep 24, 2025

Finish implementation of MLC_ENABLE_SENTENCEPIECE_TOKENIZER

Status: Open.

#84 In mlc-ai/tokenizers-cpp;

· WilliamTambellini opened on Jul 28, 2025

Upgrade to official sentencepiece 0.2

Status: Open.

#83 In mlc-ai/tokenizers-cpp;

· WilliamTambellini opened on Jul 22, 2025

Call for a new release (0.1.2?)

Status: Open.

#82 In mlc-ai/tokenizers-cpp;

· WilliamTambellini opened on Jul 17, 2025

Add support for generating dynamic library

Status: Open.

#74 In mlc-ai/tokenizers-cpp;

· PARTHSQL opened on Jun 11, 2025

rustc E0308 error when compiling version v.0.1.0

Status: Open.

#72 In mlc-ai/tokenizers-cpp;

· gaokao123 opened on May 28, 2025

Does it support Wordpiece tokenizer?

Status: Open.

#71 In mlc-ai/tokenizers-cpp;

· yangxudong opened on May 19, 2025