.NET MAUI CollectionView resets to first index

So I'm trying to do a infinite scroll both left and right. I was able to do it properly to the right but the code doesn't work as smooth to left.
If in case the user is scrolling to the right and reaches to end it will add more itens to the ObservableCollection and load more itens. If the user scrolls to the left it will insert in the 0 index a new item to create a inifinity sorted ObservableCollection. But when I insert an item to the beginning of the list the collectionview scrolls to the 0 position.

On my ViewModel:

public void UpdateCollectionView(int mag)
        {
            if (mag == 1)
            {
                for (int i = 0; i < 5; i++)
                {
                    Dates.Add(new Label() { Text = (ConvertToDate(Dates[Dates.Count - 1].Text).Date.AddDays(1).ToString()) } ) ;
                }
            }
            else if (mag == -1)
            {
                for (int i = 0; i < 5; i++)
                {
                    Dates.Insert(0, new Label() { Text = (ConvertToDate(Dates[0].Text).Date.AddDays(-1).ToString()) });
                }
            }
        }

Code behind:

    private void collectionViewDates_Scrolled(object sender, ItemsViewScrolledEventArgs e)
    {
        vm.UpdateMounth(e.CenterItemIndex);
        if ((e.LastVisibleItemIndex - 5) == (vm.Dates.Count - 6))
            vm.UpdateCollectionView(1);
        else if (e.FirstVisibleItemIndex == 0)
        {
            vm.UpdateCollectionView(-1);
        }
    }

xaml:

                    <CollectionView x:Name="collectionViewDates" ItemsSource="{Binding Dates}" SelectionChanged="CollectionView_SelectionChanged"
                                    SelectionMode="Single" SelectedItem="{Binding SelectedItem}" ItemsUpdatingScrollMode="KeepScrollOffset" Scrolled="collectionViewDates_Scrolled"
                                 ItemsLayout="HorizontalList" VerticalOptions="CenterAndExpand">
                    <CollectionView.ItemTemplate>
                        <DataTemplate>
                               
                                <Grid>
                                    <VisualStateManager.VisualStateGroups>
                                        <VisualStateGroup Name="CommonStates">
                                            <VisualState Name="Normal"></VisualState>
                                            <VisualState Name="Selected">
                                                <VisualState.Setters>
                                                    <Setter Property="BackgroundColor" Value="Transparent"></Setter>
                                                </VisualState.Setters>
                                            </VisualState>

                                        </VisualStateGroup>
                                    </VisualStateManager.VisualStateGroups>

                                    <Border ZIndex="0" Grid.Column="0" HeightRequest="50"  WidthRequest="75" BackgroundColor="#1DABF9"
                                       Padding="0" HorizontalOptions="CenterAndExpand" Margin="25,10,25,0"
                                           VerticalOptions="FillAndExpand" >
                                            <Border.StrokeShape>
                                                <RoundRectangle CornerRadius="10,10,10,10">
                                                </RoundRectangle>
                                            </Border.StrokeShape>

                                        <Label Text="{Binding Text, Converter={StaticResource StringConverter}}" TextColor="White" FontSize="18" FontAttributes="Bold" 
                                                   HorizontalTextAlignment="Center" VerticalTextAlignment="Center">
                    
                                        </Label>
                                        </Border>

                                </Grid>

                        </DataTemplate>
                    </CollectionView.ItemTemplate>
                </CollectionView>

If it reaches to the start of the CollectionView it loads 5 more itens to the left.

Example:
1 2 3 4 5
-4 -3 -2 -1 0 1 2 3 4 5

But instead of keeping visible 1 2 3 it will scroll to -4 -3 -2 since -4 now has the 0 index.

There is a way to lock the CollectionView of scrolling to the beginning of the list? or doing an inifinity scroll both ways in a different way? I really need some help to solve this or other ideia of how to make this feature.

I created a samll sample to reproduce your problem. But when I insert item at the head of the collectionview, it will not scroll to the 0 index.

Page.xaml:

<ContentPage xmlns="http://schemas.microsoft.com/dotnet/2021/maui"
             xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
             x:Class="MauiAppTest.NewPage1"
             Title="NewPage1">
    <VerticalStackLayout>
        <CollectionView x:Name="collection" ItemsLayout="HorizontalList" Scrolled="collection_Scrolled">
            <CollectionView.ItemTemplate>
                <DataTemplate>
                    <Label Text="{Binding}" HeightRequest="80" WidthRequest="80" BackgroundColor="Pink"/>
                </DataTemplate>
            </CollectionView.ItemTemplate>
        </CollectionView>
    </VerticalStackLayout>
</ContentPage>

And the page.cs:

public partial class NewPage1 : ContentPage
{
      ObservableCollection<string> values = new();
      public NewPage1()
      {
            InitializeComponent();
            for(int i = 0; i < 10; i++)
            {
                  values.Add(i.ToString());
            }
            collection.ItemsSource = values;
            
      }

      private void collection_Scrolled(object sender, ItemsViewScrolledEventArgs e)
      {
            if(e.FirstVisibleItemIndex == 0)
            {
            for (int i = 0; i < 5; i++)
            {
                values.Insert(0,i.ToString());
            }
        }
            if(e.LastVisibleItemIndex == values.Count - 1)
            {
            for (int i = 0; i < 5; i++)
            {
                values.Add(i.ToString());
            }
        }
    }
}