EF Core。批量方法。内存泄漏?

huangapple go评论104阅读模式
英文:

EF core. Bulk methods. Memory leak?

问题

I have a program that transfers a table from 1 database to another. The table contains 24k records.

I create contexts using a factory.

Program.cs

builder.Services.AddDbContextFactory<StackFlContext>(x =>
            {
                x.UseSqlServer(builder.Configuration.GetConnectionString("StackFLDb"));
                x.UseQueryTrackingBehavior(QueryTrackingBehavior.NoTracking);
            });
            builder.Services.AddDbContextFactory<StackUlContext>(x =>
            {
                x.UseSqlServer(builder.Configuration.GetConnectionString("StackULDb"));
                x.UseQueryTrackingBehavior(QueryTrackingBehavior.NoTracking);
            });
            builder.Services.AddDbContextFactory<SabpekContext>(x =>
            {
                x.UseSqlServer(builder.Configuration.GetConnectionString("SabpekDb"));
                x.UseQueryTrackingBehavior(QueryTrackingBehavior.NoTracking);
            });

I transfer the table like this:

public class IntegrationStackReferencesService : IIntegrationReferencesService
    {
        private readonly IDbContextFactory<StackFlContext> _stackFLContextFactory;
        private readonly IDbContextFactory<StackUlContext> _stackULContextFactory;
        private readonly IDbContextFactory<SabpekContext> _sabpekContextFactory;
        private readonly ILogger _logger;

        public IntegrationStackReferencesService(IDbContextFactory<StackFlContext> stackFLContextFactory,
                                                 IDbContextFactory<StackUlContext> stackULContextFactory,
                                                 IDbContextFactory<SabpekContext> sabpekContextFactory,
                                                 ILogger logger
            )
        {
            _stackFLContextFactory = stackFLContextFactory;
            _stackULContextFactory = stackULContextFactory;
            _sabpekContextFactory = sabpekContextFactory;
            _logger = logger;
        }

        public async Task SyncStackReferences()
        {
            // Each block is protected by try-catch, so if any reference fails to synchronize or there is an error,
            // methods for synchronizing others will still be executed.
            using (var stackFlContext = _stackFLContextFactory.CreateDbContext())
            using (var stackUlContext = _stackULContextFactory.CreateDbContext())
            using (var sabpekContext = _sabpekContextFactory.CreateDbContext())
            {
                await SyncTechStructureAsync<Models.Stack.StackFl.ВозможнаяСтруктура>(stackFlContext, sabpekContext);
            }
        }
private async Task SyncTable<T>(DbContext dbContext, IQueryable<T> list) where T : class
        {
            dbContext.Set<T>().Truncate();
            dbContext.Set<T>().BulkInsert(list);
            //await dbContext.SaveChangesAsync();
        }

As a result, before the start of the transfer, I have the following memory consumption enter image description here

After transfer: enter image description here

I expect the memory should be cleared to +- first value, no? what am i doing wrong? Contexts should have been cleaned up after "using". Why the memory is not cleared and what should I do to clear it?

英文:

I have a program that transfers a table from 1 database to another. The table contains 24k records.

I create contexts using a factory.

Program.cs

builder.Services.AddDbContextFactory&lt;StackFlContext&gt;(x =&gt;
            {
                x.UseSqlServer(builder.Configuration.GetConnectionString(&quot;StackFLDb&quot;));
                x.UseQueryTrackingBehavior(QueryTrackingBehavior.NoTracking);
            });
            builder.Services.AddDbContextFactory&lt;StackUlContext&gt;(x =&gt;
            {
                x.UseSqlServer(builder.Configuration.GetConnectionString(&quot;StackULDb&quot;));
                x.UseQueryTrackingBehavior(QueryTrackingBehavior.NoTracking);
            });
            builder.Services.AddDbContextFactory&lt;SabpekContext&gt;(x =&gt;
            {
                x.UseSqlServer(builder.Configuration.GetConnectionString(&quot;SabpekDb&quot;));
                x.UseQueryTrackingBehavior(QueryTrackingBehavior.NoTracking);
            });

I transfer the table like this:

public class IntegrationStackReferencesService : IIntegrationReferencesService
    {
        private readonly IDbContextFactory&lt;StackFlContext&gt; _stackFLContextFactory;
        private readonly IDbContextFactory&lt;StackUlContext&gt; _stackULContextFactory;
        private readonly IDbContextFactory&lt;SabpekContext&gt; _sabpekContextFactory;
        private readonly ILogger _logger;

        public IntegrationStackReferencesService(IDbContextFactory&lt;StackFlContext&gt; stackFLContextFactory,
                                                 IDbContextFactory&lt;StackUlContext&gt; stackULContextFactory,
                                                 IDbContextFactory&lt;SabpekContext&gt; sabpekContextFactory,
                                                 ILogger logger
            )
        {
            _stackFLContextFactory = stackFLContextFactory;
            _stackULContextFactory = stackULContextFactory;
            _sabpekContextFactory = sabpekContextFactory;
            _logger = logger;
        }

        public async Task SyncStackReferences()
        {
            //Каждый блок защищён try catch поэтому если какой-то справочник не синхронизируется или будет ошибка,
            //то методы синхронизации других, всё равно будут выполнены
            using (var stackFlContext = _stackFLContextFactory.CreateDbContext())
            using (var stackUlContext = _stackULContextFactory.CreateDbContext())
            using (var sabpekContext = _sabpekContextFactory.CreateDbContext())
            {
                await SyncTechStructureAsync&lt;Models.Stack.StackFl.ВозможнаяСтруктура&gt;(stackFlContext, sabpekContext);
            }
        }
private async Task SyncTable&lt;T&gt;(DbContext dbContext, IQueryable&lt;T&gt; list) where T : class
        {
            dbContext.Set&lt;T&gt;().Truncate();
            dbContext.Set&lt;T&gt;().BulkInsert(list);
            //await dbContext.SaveChangesAsync();
        }

As a result, before the start of the transfer, I have the following memory consumption
enter image description here

After transfer:

enter image description here

I expect the memory should be cleared to +- frist value, no? what am i doing wrong? Contexts should have been cleaned up after "using". Why the memory is not cleared and what should I do to clear it?

答案1

得分: 1

这绝对不是内存泄漏的迹象。

.NET中的垃圾回收(GC)并不会持续释放内存。以下是文档中的一句引用:

垃圾回收在以下情况之一为真时发生:

-- 系统的物理内存较低。内存大小由操作系统的低内存通知或主机指示的低内存检测。

-- 托管堆上分配对象使用的内存超过了可接受的阈值。此阈值会随着进程的运行不断调整。

我建议你在这里阅读更多关于GC的信息:链接

在你的示例中,内存消耗仅增加到了127MB。在任何现代硬件上,这都太少了,GC不会认为值得去做额外的工作。

英文:

This is not a sign of memory leak at all.

Garbage collection (GC) in .NET is not constantly freeing memory. A quote from the documentation:

> Garbage collection occurs when one of the following conditions is
> true:
>
> -- The system has low physical memory. The memory size is detected by
> either the low memory notification from the operating system or low
> memory as indicated by the host.
>
> -- The memory that's used by allocated objects on the managed heap
> surpasses an acceptable threshold. This threshold is continuously
> adjusted as the process runs.

I suggest you read more about GC here.

The memory consumption in your example has increased to merely 127 MB.
This is too little on any modern hardware and is not considered worth the effort by GC.

huangapple
  • 本文由 发表于 2023年7月6日 16:37:11
  • 转载请务必保留本文链接:https://go.coder-hub.com/76626975.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定