Splitting a UTF-8 string into chunks

user_450989 0
Viewed 486

I want to split a UTF-8 string into chunks of equal size. I came up with a solution that does exactly that. Now I want to simplify it removing the first collect call if possible. Is there a way to do it?

fn main() {
    let strings = "ĄĆĘŁŃÓŚĆŹŻ"
        .chars()
        .collect::<Vec<char>>()
        .chunks(3)
        .map(|chunk| chunk.iter().collect::<String>())
        .collect::<Vec<String>>();
    println!("{:?}", strings);
}

Playground link

rust iterator string
3 Answers

You can use chunks() from Itertools.

use itertools::Itertools; // 0.10.1

fn main() {
    let strings = "ĄĆĘŁŃÓŚĆŹŻ"
        .chars()
        .chunks(3)
        .into_iter()
        .map(|chunk| chunk.collect::<String>())
        .collect::<Vec<String>>();
    println!("{:?}", strings);
}
user_2345900

This doesn't require Itertools as a dependency and also does not allocate, as it iterates over slices of the original string:

fn chunks(s: &str, length: usize) -> impl Iterator<Item=&str> {
    assert!(length > 0);
    let mut indices = s.char_indices().map(|(idx, _)| idx).peekable();
    
    std::iter::from_fn(move || {
        let start_idx = match indices.next() {
            Some(idx) => idx,
            None => return None,
        };
        for _ in 0..length - 1 {
            indices.next();
        }
        let end_idx = match indices.peek() {
            Some(idx) => *idx,
            None => s.bytes().len(),
        };
        Some(&s[start_idx..end_idx])
    })
}


fn main() {
    let strings = chunks("ĄĆĘŁŃÓŚĆŹŻ", 3).collect::<Vec<&str>>();
    println!("{:?}", strings);
}
user_27229680

Having considered the problem with graphemes I ended up with the following solution.

I used the unicode-segmentation crate.

use unicode_segmentation::UnicodeSegmentation;                                                                                                                            

fn main() {
    let strings = "ĄĆĘŁŃÓŚĆŹŻèèèèè"
        .graphemes(true)                                                                                                                                          
        .collect::<Vec<&str>>()                                                                                                                                   
        .chunks(length)                                                                                                                                           
        .map(|chunk| chunk.concat())                                                                                                                              
        .collect::<Vec<String>>();
    println!("{:?}", strings);
}

I hope some simplifications can still be made.

user_4509890
Related