How to reduce the time of calculations in VB.net?

I have to perform some calculations for almost 5000 items. I have used parallel for loop and the time taken for these calculations is almost 5 sec. I want to do it in below 1 sec. Any suggestions will be highly appreciated.
The code is this

Dim AverageVal As New Double
Dim maxVal, minVal As New Double
Dim error_ As New Double
Dim delta As New List(Of Double)
With gOutputStresses_LC(0)
Dim UniqueTagsList As List(Of Integer) = gOutputStresses_LC(0).stresses.Select(Function(c) c.Tag0D).Distinct.ToList
Parallel.ForEach(UniqueTagsList, Sub(Uniq)
Dim tstresses As New List(Of clsMSHStress)
tstresses = .stresses.FindAll(Function(s) s.Tag0D = Uniq)
tstresses.Sort(Function(x, y) x.Sxx.CompareTo(y.Sxx))
tstresses.Reverse()
'maxVal = tstresses(0).Sxx
'minVal = tstresses.Last.Sxx
error_ = Math.Abs((tstresses(0).Sxx - tstresses.Last.Sxx) / (tstresses(0).Sxx + tstresses.Last.Sxx))
If error_ < 10 Then
AverageVal = tstresses.Sum(Function(s) s.Sxx) / tstresses.Count
.stresses.FindAll(Function(s) s.Tag0D = Uniq).ForEach(Sub(c) c.Sxx = AverageVal)
Else
AverageVal = 0
delta = New List(Of Double)
For k As Integer = 1 To tstresses.Count - 1
delta.Add(tstresses(k).Sxx - tstresses(k - 1).Sxx)
Next
Dim idx As Integer = delta.FindIndex(Function(c) c = delta.Max)
For g As Integer = 0 To idx - 1
AverageVal += tstresses(g).Sxx
Next
AverageVal = AverageVal / idx
For g As Integer = 0 To idx - 1
tstresses(g).Sxx = AverageVal
Next
AverageVal = 0
For g As Integer = idx To tstresses.Count - 1
AverageVal += tstresses(g).Sxx
Next
AverageVal = AverageVal / (tstresses.Count - idx)
For g As Integer = idx To tstresses.Count - 1
tstresses(g).Sxx = AverageVal
Next
For p As Integer = 0 To tstresses.Count - 1
''.Stresses.Where(Function(s) s.Tag0D = tstresses(p).Tag0D And s.Tag2D = tstresses(p).Tag2D) = tstresses(p)
'Dim idx As Integer =
.stresses(.stresses.FindIndex(Function(s) s.Tag0D = tstresses(p).Tag0D And s.Tag2D = tstresses(p).Tag2D)) = tstresses(p)
Next
End If
End Sub)
End With    

The UniqueTagsList contains almost 5000 items.

5000 items is nothing if the algorithm is done correctly. We can probably do this in less than a second without needing to go parallel at all. Given a modern commodity CPU clocked at a modest, say, 2 GHz, that's 2 billion clock cycles. Even better, you probably get at least 3 instructions per tick. That's 6 billion total. Simple division means we can fit this in a single second on a single non-parallel core as long as we're spending less than 1.2 million instructions per item. In fact, memory latency is probably our limiting factor here, and parallel execution might not even fix that.

A significant savings potential from the original is this loop (and similar):

For p As Integer = 0 To tstresses.Count - 1
    ''.Stresses.Where(Function(s) s.Tag0D = tstresses(p).Tag0D And s.Tag2D = tstresses(p).Tag2D) = tstresses(p)
    'Dim idx As Integer =
    .stresses(.stresses.FindIndex(Function(s) s.Tag0D = tstresses(p).Tag0D And s.Tag2D = tstresses(p).Tag2D)) = tstresses(p)
Next

It seems to be trying to match up each tstresses item with the same item from the original gOutputStresses_LC array. But since it also appears we are working with reference types (the cls prefix is not typically used for Structure), we can know these are already the same object, and the lookups back into the original array are not needed. All this will ever do is waste a bunch of time setting each array element to the same value it already has.

There is similar code elsewhere running FindIndex() or FindAll() doing extra lookups back in an earlier array. This extra work is significant and not needed, converting something that would be close to O(n) (after the sort) to something that's instead O(n²).

Once we understand this, the simplification opens up other avenues for improvement as well. In short, the code below should be MUCH faster by avoiding the creation of extra needless Lists/arrays and by significantly reducing the number of passes through the data.

I need to add a disclaimer that I typed this directly into the reply window, and without the benefit of any sample data to test against. It's likely there's a bug or two you'll still need to work through, including a potential off-by-one error around the final IdxAtMaxDelta break, so be sure to test thoroughly.

Dim TagGroups = gOutputStresses_LC(0).stresses.GroupBy(Function(c) c.Tag0D)
For Each grp In TagGroups
Dim sorted = grp.OrderByDesc(Function(i) i).ToList()
Dim last = sorted(sorted.Count - 1)
Dim error_ As Double = Math.Abs( (sorted(0).Sxx - last.Sxx) / 
(sorted(0).Sxx + last.Sxx) )
If error_ < 10.0 Then
Dim AverageVal As Double = sorted.Sum(Function(s) s.Sxx) / sorted.Count
For Each item In grp
item.Sxx = AverageVal
Next
Else
' Only need 2 loops through the data and NO NESTED PASSES
' (original code used 4 base passes, of which 2 had nested passes)
Dim Total As Double = 0.0
Dim idx As Integer = 0
Dim prior As clsMSHStress = sorted(0)
Dim MaxDelta As Integer = 0.0
Dim IdxAtMaxDelta As Integer = 1
Dim TotalAtMaxDelta As Double = 0.0
' Loop 1
For Each item In sorted.Skip(1)
total += item.Sxx
idx += 1
Dim delta As Double = item.Vss - prior.Vss
prior = s
If delta > MaxDelta Then
MaxDelta = delta
MaxIdx = idx
TotalAtMaxDelta = total
End If
Next
Dim lowAverage As Double = TotalAtMaxDelta / IdxAtMaxDelta
Dim highAverage As Double = (Total - TotalATMaxDelta) /
(sorted.Count - IdxAtMaxDelta)
' Loop 2 (first half)
Dim i As Integer = 0
For i = 0 To IdxAtMaxDelta
sorted(i).Vss = lowAverage
Next
' Loop 2 (second half)
For i = i To sorted.Count - 1
sorted(i).Vss = highAverage
Next
End If
Next