SQL Server running expensive Look up and aggregations then insert the result in to new table

by Hang Yang   Last Updated October 09, 2019 14:06 PM - source

There are 4 tables containing about 100 M rows each, they are the tables I will look up via a column(eg. ID) in all 4 tables, we have the 5th table, which has 18M data with the(ID and SID) 1 to 1 relationship, each ID maps to about 500 records in the other 4 tables. Now I am fetching minimal date from the 4 tables with each SID and insert them in a new table.

The logic is simple but it is really time consuming so i am trying to use parallel.for() from .net. Breaking the 18 M table into 180 groups and run them concurrently, but this is limited by the hardware, I wonder if there is better way to do it.

Related Questions

creating index on big table(s)

Updated September 03, 2018 18:06 PM

Linked Server Error on SQL Agent Job

Updated March 16, 2016 08:02 AM