Navigation in WPF searchable combobox

I've got my custom ComboBox:

public class CustomComboBox : ComboBox
{
    protected override void OnSelectionChanged(SelectionChangedEventArgs e)
    {
        // allow to go into items using up and down arrows without causing the text change
        e.Handled = true;
    }
}

and xaml as:

<Window>
    <Grid>
        <wpfApplication1:CustomComboBox IsEditable="True" 
                  Width="200" 
                  Height="25" 
                  IsTextSearchEnabled="False" 
                  StaysOpenOnEdit="True"
                  x:Name="cb" 
                  PreviewTextInput="Cb_OnPreviewTextInput" 
                  ItemsSource="{Binding Projects}" 
                  Text="{Binding Text}"
                  SelectionChanged="Cb_OnSelectionChanged">
            <ComboBox.ItemsPanel>
                <ItemsPanelTemplate>
                    <VirtualizingStackPanel />
                </ItemsPanelTemplate>
            </ComboBox.ItemsPanel>
        </wpfApplication1:CustomComboBox>
    </Grid>
</Window>

Code of the window:

public partial class MainWindow : INotifyPropertyChanged
{
    public ICollectionView Projects { get; set; }

    public MainWindow()
    {
        InitializeComponent();
        DataContext = this;

        ObservableCollection = new ObservableCollection<string>(Enumerable.Range(1, 1000).Select(i => $"Item {i}"));

        Projects = CollectionViewSource.GetDefaultView(ObservableCollection);
    }

    private string _text;
    public ObservableCollection<string> ObservableCollection { get; set; }

    public string Text
    {
        get { return _text; }
        set
        {
            if (_text != value)
            {
                _text = value;
                OnPropertyChanged();

                Projects.Filter = f =>
                {
                    var search = Text.ToLower();
                    var item = f.ToString().ToLower();

                    return item.Contains(search);
                };
            }
        }
    }

    private void Cb_OnPreviewTextInput(object sender, TextCompositionEventArgs e)
    {
        cb.IsDropDownOpen = true;
    }

    public event PropertyChangedEventHandler PropertyChanged;

    [NotifyPropertyChangedInvocator]
    protected virtual void OnPropertyChanged([CallerMemberName] string propertyName = null)
    {
        PropertyChanged?.Invoke(this, new PropertyChangedEventArgs(propertyName));
    }
}

When Cb_OnPreviewTextInput is called I set IsDropdownOpen = true. In the first attempt (after typing the first letter) the first item on the list is selected and I can go up and down using relevant arrows, the caret is still in the TextBox.

When I keep on typing at that point, I',m not able to navigate up and down (1 item at time) anymore, at this point the whole ScrollViewer gets focus and I can only go to the bottom or to the top, but not 1 by 1. I have to close the popup e.g. by pressing Escape and then reopen by typing 1 character to be able to go up/down again.

I also noticed that after pressing PageUp the first item gets selected as well, so I tried to mimic that in code, but no luck.

Anyone knows what to do here to be able to navigate up/down and type without problems?

I tried things like:

protected override void OnPreviewKeyDown(KeyEventArgs e)
{
    if (_popup != null && e.Key == Key.Up || e.Key == Key.Down)
    {
        Dispatcher.BeginInvoke(DispatcherPriority.Input, 
        new Action(() =>
        {
            var c = _popup.Child;
            var vs = c.GetChildOfType<VirtualizingStackPanel>();
            vs.PageUp();
            vs.Focus();
            Keyboard.Focus(vs);
        }));
    }

    base.OnPreviewKeyDown(e);
}

But that didn't work.