big-data-Application.docx

《big-data-Application.docx》由会员分享，可在线阅读，更多相关《big-data-Application.docx（5页珍藏版）》请在优知文库上搜索。

1、ApplicationofbigdataAbstract:Sincethemid1980s.theworldhasexperiencedanunprecedentedexplosioninthecapacitytoproduce.More,andcommunicaccdata,primarilyindigitalfmuts.Simultaneously,accesstocomputingtechnologiesintheformof(hepersonalPC.SnlarlPhone.andotherhandhelddeviceshasmirroredIhisgrowth.Wiihlhcscen

2、hancedCapabililiesofdatastorageandrapidcompulationaswellasreal-timedeliveryofinformationviatheinternet,theaveragedailyconsumpwercompaniesand(heircustomersconserveenergybygivingfeedbackmessagesFromamalfunctioningmetertothepowercompanyoralarmingcustomerswhentheyarcwastingenergy.CrimePredictionBigDataa

3、nalyticsishelpingPOliCCdepartmentstodeveloppredictionalgorithmstopreventprobableCrimCcommitments.TechnologyHadoopEcosystemVolume:largeamountofdataVariety:heterogeneityofdatatypes,representation,andsemanticinterpretation.Velocity:boththerateatwhichdataarriveandthetimeinwhichitmustbeactedupon.Veracity

4、:referstothebiases,noiseandabnormalityindata.Isthedatathatisbeingstored,andminedmeaningfultotheproblembeinganalyzed.(Inscopingoutyourbigdatastrategyyouneedtohaveyourteamandpartnersworktohelpkeepyourdatacleanandprocessestokeep,dirtydata,fromaccumulatinginyoursystems.)Challenges:Privacy:BiSDQIaprofoun

5、dlychangextheprMenofprivacyanotherreasonwhyIheUS,xdata-drivenCtvnpaniesareM)byinfsohardtousethereviewofdataprotectionlawtoweakenit.AbigdataSOCiCIyseemslbeinevilable.andPrVJniSCSmuch,butprivacy(pnperlyImderSkKMI)mustbeanimportantpartofanysuchsociety.First,weneedtthinkbroadlyaboutprivacyasmorethanjust

6、(hekeepingofasecret,butastherulesthatmustgovernPerSOnalinformation.Second,weneedUrealize(hatinformationdoesnlloselegalPrUkXtionjuslbecauseitisheldbyanotherperson.Confidentialityruleswillbeessentialinanyb设datafuture.Ttiini.wcneedtodevelopsomebigdataethicsasasociety.BigdatainthefieldofEducation:Studei

7、nsusingonlinesystemSystemtocollectandrecordtheStUdCnlSonlinelearningbehavior.savinginlodatabaseFordataanalysisandprocessing,predictingthestudents,academicperformanceTheresultoftbcforecastandfeedbackforvisualprocessingToniecetherequirvmentsofthestudents*individuallearningmaterialsTeacher.dminisirator

8、s.anddeveloperstimelytogivestudenlsBUkklnCCandhelpBigdatahaschangedthewayfinancialinstitutionsoperateintraditionaldatafromfourdifferentways，achievinggreatbusinessvalue。Thesefourareas(4C)include：CompatibilityConnectedness*Cost*Capitalization.Theapplicationoflargedatainfinancialindustryisgraduallyexpa

9、nding,Overseas,bigdatahavebeenfullytriedinthefinancialindustry,suchasriskcontrol,operationmanagementalessupportandbusinessmodelinnovation.InChina,theapplicationoflargedatainfinancialinstitutionsisstillbasicallyattheinitialstage.Keychallengessuchasdataconsolidationanddepartmentalcoordinationremainthe

10、mainbottlenecksthathinderfinancialinstitutionsfromtranslatingdataintovalue.Example:AnonlinestorehasanexistingSQ1.Databasethatmanagesitsauthorizationsystemandstorcf11Hit.SQ1.isnecessaryinordertoensurenolosttransactionsandfullACIDcompliance.Thestorealso(racksitssocialaclivilyusingaKEY/Valuedalabasc.Th

11、isdalabasePrwiIkSspeedandcaneasilystorelargeamountsofdatabutPrOVidCSlessconsistencythanaSQ1.SOIUtion.However,associaldataisnoascriticalastransactional,thissolutionisappropriate.Thestorealsoprovidesrvconuncndalionstocustomersaboutproductstheymightlike.TheyuseaColumndatabaseforthispurpose.SQ1.isnotncc

12、essar)becausetheinformationisnotrelational.TheMinistryofEducationisresponsibleforInanagingdataaboutstudentsaround(heprovince.Theyhaveover125millionrecords(students).241millionrowsandover15yearsworthofdata.hcdataisstoredusing450tables,buttheonlyvarietyisatattributelcvcl(i.c.StandardizedtcMing,pcrtbnn

13、ancc.ccc).Theyreceivethedatafromindividualschoolsingiantchunksfourlimesayear.TheMinistrydeckledagainstusingbigdatatoolstomanagetheirdataandisstillusingaRDBMS.Ahhoughtheyhavalareamoun(sofdata,the(yeoftheirdataandIbevelocityatwhichIheyreceiveitmeansthataRDBMSisstillbest.TheyalsohavecomplianceandprivacyissuesthatmakeNoSQ1.anunsuitablechoice.EachschoolandschoolboardwassendingthedataoIheMinistryusingdifferentschemasand(hisinconsistencymadedatamanagementdifikuhanddecreasedthevalueofIhedataforBipurposes.TheMinistrylookamasterdatamanagementapproach.Overthecourseof4years,theycreatedcommonmclhgiesa