Quantcast
Channel: Visual C# forum
Viewing all articles
Browse latest Browse all 31927

Load large nested xml file into sql server tables using Custom XMLTextReader and SQLBulkcopy ( IDataReader).

$
0
0

Hi,
i have insert a huge xml file (2 GB size) to sql server database. The xml file is nested with child elements. The data should go to 3 different tables. I searxhed on internet and arrived at usingXMLTextReader and SQLBulkcopy.

Since my xml is nested, somebody mentioned to implement a custom IDataReader. But i am not able to find a correct sample code to achieve this.

How do i read the xml element values to 3 different tables. I was planning to implement 3 sqlbulkcopy implementations

for 3 different IDataReaders ( 1 for each sub-node types). ie reading the xml 3 times and extract the target table data 3 times.

My main concern is how do i map the innder nodes of xml file to get the column values for the destination table.

XML Format :
------------------
<root_elementroot_element>

<FileMetaData>

<FileName>root_element_201312020304.XML</FileName>

<FileID>312020304</FileID>

<SenderID>571299948</SenderID>

<SentDate>20131202</SentDate>

<SentTime>03:04:00</SentTime>

<ReceiverID></ReceiverID>

</FileMetaData>

<Sender>

<Address>

<PrimaryStreet>123 street</PrimaryStreet>

<TaxID>1099948</TaxID>

<InsurerName></InsurerName>

<InsurerID></InsurerID>

<Sponsors>

<Sponsor>

<Address>

<PrimaryStreet>200 1st st</PrimaryStreet>

</Address>

<Name>abc Foundation for testResearch</Name>

<TaxID>4115ss0</TaxID>

<Contracts>

<Contract>

<Code_Se>112478514</Code_Se>

<Metadata>

<TransactionType>ADD</TransactionType>

</Metadata>

<Member> 

<Metadata>

<TransactionType>ADD</TransactionType>

</Metadata>

<Address><PrimaryStreet>77 Broadway</PrimaryStreet></Address>

<FirstName>fnametest</FirstName>

<LastName>lnametest</LastName>

<Relationship>SUBSCRIBER</Relationship>

<PayrollID>300074</PayrollID>

<rcode>112478514</rcode>

<Gender>F</Gender>

<PersonType>S</PersonType>

<BirthDate>1230101</BirthDate>

<MaritalStatus>SINGLE</MaritalStatus>

<Ethnicity>WH</Ethnicity>

<EffectiveChangeDate>20131018</EffectiveChangeDate>

<EmploymentStatus>FT</EmploymentStatus>

<BenefitStatus>ACCEPTED</BenefitStatus>

<Benefits>

<Benefit BenefitType='ADD'>

<TransactionType>AD</TransactionType>

<CoverageIndicator>true</CoverageIndicator>

<CoverageAmount>116000</CoverageAmount>

<OriginalEffectiveDate>20131023</OriginalEffectiveDate>

<CoverageEffectiveDate>20131023</CoverageEffectiveDate>

<!--second child table, t_member_benefit_carrieridentifier  starts from here  -->

<CarrierIdentifiers><CarrierIdentifier>

<Value>E168</Value>

<EffectiveDate>20131023</EffectiveDate>

<Name>PAYROLL_ATTRIBUTE_3</Name>

</CarrierIdentifier>

<CarrierIdentifier>

<Value>45203</Value>

<EffectiveDate>20131023</EffectiveDate><Name>CONTROL_NBR</Name>

</CarrierIdentifier><CarrierIdentifier><EffectiveDate>20131023</EffectiveDate>

<Name>PAYROLL_ATTRIBUTE_2</Name></CarrierIdentifier><CarrierIdentifier>

<EffectiveDate>20131023</EffectiveDate><Name>PAYROLL_ATTRIBUTE_1</Name>

</CarrierIdentifier></CarrierIdentifiers>

<EnrollmentStatus>TIMELY</EnrollmentStatus>

<SalaryMultiplier>01000000</SalaryMultiplier><Premium>

<TotalCost>27.84</TotalCost><SponsorCost>27.84</SponsorCost>

<MemberCost>0.00</MemberCost>

<TotalBenefitProgramAmount>0.00</TotalBenefitProgramAmount>

<!-- 3rrd table, t_member_categories nodes starts from here....-->

 - <Categories>
- <Category>
  <Value>N</Value>
  <EffectiveDate>20131121</EffectiveDate>
  <Name>ASD before 1/1/11</Name>
  </Category>
- <Category>
  <Value>N</Value>
  <EffectiveDate>20131121</EffectiveDate>
  <Name>Linked Spouse</Name>
  </Category>
  etc etc
  </Categories>

</Premium></Benefit>
</member></contract>
----------------------
------------------------
etc etx
 </Contracts>
</Sponsor>
</Sponsors>
</Sender>
</root_element>

Destination tables
-------------------------

1.t_main_member_table
2.t_member_benefit_carrieridentifier  /* first set of child nodes */
2.t_member_categories  /* second set of child nodes */


Viewing all articles
Browse latest Browse all 31927

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>