Installation error `ERROR: Cannot install tortoise-tts and tortoise-tts==3.0.0 because these package versions have conflicting dependencies.`

[aserebryakov]

I am trying to install tortoise on Windows 11 WSL (Ubuntu 22.04) using pip install git+https://github.com/neonbjb/tortoise-tts

Installation in clean pipenv environment.

Please see the log below:

Collecting git+https://github.com/neonbjb/tortoise-tts
  Cloning https://github.com/neonbjb/tortoise-tts to /tmp/pip-req-build-6t5hw3ls
  Running command git clone --filter=blob:none --quiet https://github.com/neonbjb/tortoise-tts /tmp/pip-req-build-6t5hw3ls
  Resolved https://github.com/neonbjb/tortoise-tts to commit 8a2563ecabe93c4fb626f876dd0c52c966edef2f
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting tqdm (from tortoise-tts==3.0.0)
  Downloading tqdm-4.67.1-py3-none-any.whl.metadata (57 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.7/57.7 kB 640.3 kB/s eta 0:00:00
Collecting rotary_embedding_torch (from tortoise-tts==3.0.0)
  Downloading rotary_embedding_torch-0.8.6-py3-none-any.whl.metadata (675 bytes)
Collecting inflect (from tortoise-tts==3.0.0)
  Downloading inflect-7.5.0-py3-none-any.whl.metadata (24 kB)
Collecting progressbar (from tortoise-tts==3.0.0)
  Downloading progressbar-2.5.tar.gz (10 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting einops (from tortoise-tts==3.0.0)
  Downloading einops-0.8.1-py3-none-any.whl.metadata (13 kB)
Collecting unidecode (from tortoise-tts==3.0.0)
  Downloading Unidecode-1.4.0-py3-none-any.whl.metadata (13 kB)
Collecting scipy (from tortoise-tts==3.0.0)
  Downloading scipy-1.15.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (61 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.0/62.0 kB 2.7 MB/s eta 0:00:00
Collecting librosa (from tortoise-tts==3.0.0)
  Downloading librosa-0.11.0-py3-none-any.whl.metadata (8.7 kB)
Collecting transformers==4.31.0 (from tortoise-tts==3.0.0)
  Downloading transformers-4.31.0-py3-none-any.whl.metadata (116 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 116.9/116.9 kB 3.6 MB/s eta 0:00:00
Collecting tokenizers==0.14.0 (from tortoise-tts==3.0.0)
  Downloading tokenizers-0.14.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.7 kB)
Collecting scipy (from tortoise-tts==3.0.0)
  Downloading scipy-1.13.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (60 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.6/60.6 kB 2.8 MB/s eta 0:00:00
Collecting numpy<2.3,>=1.22.4 (from scipy->tortoise-tts==3.0.0)
  Downloading numpy-2.2.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (62 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.0/62.0 kB 3.6 MB/s eta 0:00:00
Collecting huggingface_hub<0.17,>=0.16.4 (from tokenizers==0.14.0->tortoise-tts==3.0.0)
  Downloading huggingface_hub-0.16.4-py3-none-any.whl.metadata (12 kB)
Collecting filelock (from transformers==4.31.0->tortoise-tts==3.0.0)
  Downloading filelock-3.18.0-py3-none-any.whl.metadata (2.9 kB)
Collecting packaging>=20.0 (from transformers==4.31.0->tortoise-tts==3.0.0)
  Downloading packaging-25.0-py3-none-any.whl.metadata (3.3 kB)
Collecting pyyaml>=5.1 (from transformers==4.31.0->tortoise-tts==3.0.0)
  Downloading PyYAML-6.0.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.1 kB)
Collecting regex!=2019.12.17 (from transformers==4.31.0->tortoise-tts==3.0.0)
  Downloading regex-2024.11.6-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (40 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.5/40.5 kB 2.6 MB/s eta 0:00:00
Collecting requests (from transformers==4.31.0->tortoise-tts==3.0.0)
  Downloading requests-2.32.3-py3-none-any.whl.metadata (4.6 kB)
INFO: pip is looking at multiple versions of transformers to determine which version is compatible with other requirements. This could take a while.
ERROR: Cannot install tortoise-tts and tortoise-tts==3.0.0 because these package versions have conflicting dependencies.

The conflict is caused by:
    tortoise-tts 3.0.0 depends on tokenizers==0.14.0
    transformers 4.31.0 depends on tokenizers!=0.11.3, <0.14 and >=0.11.1

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

[conradopsa]

Hi!

I've solved replacing the tokenizer version inside requirements.txt and setup.py to tokenizers==0.13.3.

So you can try:

conda activate tortoise \
&& sed -i 's/tokenizers[^*]*/tokenizers==0.13.3/' requirements.txt \
&& sed -i 's/tokenizers[^'"'"'"]*/tokenizers==0.13.3/' setup.py \
&& pip install .

[aserebryakov]

@conradopsa , thanks, but now I have this error when building tokenizers. Which rustc version should be used and where do I configure it?

4bdeec63344e8bc61510a770eb094/target/release/build/esaxx-rs-bdf2b3384a24df2b/out -L native=/tmp/pip-install-ksjrg0aw/tokenizers_2774bdeec63344e8bc61510a770eb094/target/release/build/onig_sys-30889286838e258d/out`
      warning: variable does not need to be mutable
         --> tokenizers-lib/src/models/unigram/model.rs:265:21
          |
      265 |                 let mut target_node = &mut best_path_ends_at[key_pos];
          |                     ----^^^^^^^^^^^
          |                     |
          |                     help: remove this `mut`
          |
          = note: `#[warn(unused_mut)]` on by default

      warning: variable does not need to be mutable
         --> tokenizers-lib/src/models/unigram/model.rs:282:21
          |
      282 |                 let mut target_node = &mut best_path_ends_at[starts_at + mblen];
          |                     ----^^^^^^^^^^^
          |                     |
          |                     help: remove this `mut`

      warning: variable does not need to be mutable
         --> tokenizers-lib/src/pre_tokenizers/byte_level.rs:200:59
          |
      200 |     encoding.process_tokens_with_offsets_mut(|(i, (token, mut offsets))| {
          |                                                           ----^^^^^^^
          |                                                           |
          |                                                           help: remove this `mut`

      error: casting `&T` to `&mut T` is undefined behavior, even if the reference is unused, consider instead using an `UnsafeCell`
         --> tokenizers-lib/src/models/bpe/trainer.rs:526:47
          |
      522 |                     let w = &words[*i] as *const _ as *mut _;
          |                             -------------------------------- casting happened here
      ...
      526 |                         let word: &mut Word = &mut (*w);
          |                                               ^^^^^^^^^
          |
          = note: for more information, visit <https://doc.rust-lang.org/book/ch15-05-interior-mutability.html>
          = note: `#[deny(invalid_reference_casting)]` on by default

      warning: `tokenizers` (lib) generated 3 warnings
      error: could not compile `tokenizers` (lib) due to 1 previous error; 3 warnings emitted

[dioramayuanito]

#860 (comment)