Moment generating function | Definition, properties ...

Discover how the moment generating function (mgf) is defined. ... The next example shows how the mgf of an exponential random variable is calculated. StatLect Index>Fundamentalsof probability Momentgeneratingfunction byMarcoTaboga,PhD Themomentgeneratingfunction(mgf)isafunctionoftenusedtocharacterize thedistributionofarandomvariable. Tableofcontents HowitisusedDefinitionExampleDerivingmomentswiththemgfCharacterizationofadistributionviathemomentgeneratingfunctionMoredetailsMomentgeneratingfunctionofalineartransformationMomentgeneratingfunctionofasumofmutuallyindependentrandomvariablesMultivariategeneralizationSolvedexercisesExercise1Exercise2Exercise3References Howitisused Themomentgeneratingfunctionhasgreatpracticalrelevancebecause: itcanbeusedtoeasilyderivemoments;its derivativesatzeroareequaltothemomentsoftherandomvariable; aprobabilitydistributionisuniquelydeterminedbyitsmgf. Fact2,coupledwiththeanalyticaltractabilityofmgfs,makesthemahandy toolforsolvingseveralproblems,suchasderivingthedistributionofasum oftwoormorerandomvariables. Definition Thefollowingisaformaldefinition. Definition Let bearandomvariable.Iftheexpectedvalue existsandisfiniteforallrealnumbers belongingtoaclosedinterval , with , thenwesaythat possessesamomentgeneratingfunctionandthe functionis calledthemomentgeneratingfunctionof . Notallrandomvariablespossessamomentgeneratingfunction.However,all randomvariablespossessacharacteristic function,anothertransformthatenjoyspropertiessimilartothose enjoyedbythemgf. Example Thenextexampleshowshowthemgfofanexponential randomvariableiscalculated. Example Let beacontinuousrandomvariablewith supportand probabilitydensity functionwhere isastrictlypositivenumber.Theexpectedvalue canbecomputedas follows:Furthermore, theaboveexpectedvalueexistsandisfiniteforany , provided . Asaconsequence, possessesa mgf: Derivingmomentswiththemgf Themomentgeneratingfunctiontakesitsnamebythefactthatitcanbeused toderivethemomentsof , asstatedinthefollowingproposition. Proposition Ifarandomvariable possessesamgf , thenthe -th momentof , denotedby , existsandisfiniteforany . Furthermore,where isthe -th derivativeof withrespectto , evaluatedatthepoint . Proof Provingtheabovepropositionisquite complicated,becausealotofanalyticaldetailsmustbetakencareof(see e.g.Pfeiffer-2012).Theintuition,however,is straightforward.Sincetheexpectedvalueisalinearoperatorand differentiationisalinearoperation,underappropriateconditionswecan differentiatethroughtheexpected value:Making thesubstitution , we obtain Thenextexampleshowshowthispropositioncanbeapplied. Example Inthepreviousexamplewehavedemonstratedthatthemgfofanexponential randomvariable isThe expectedvalueof canbecomputedbytakingthefirstderivativeofthe mgf:and evaluatingitat :The secondmomentof canbecomputedbytakingthesecondderivativeofthe mgf:and evaluatingitat :And soonforhighermoments. Characterizationofadistributionviathemoment generatingfunction Themostimportantpropertyofthemgfisthefollowing. Proposition Let and betworandomvariables.Denoteby and theirdistribution functionsandby and theirmgfs. and havethesamedistribution(i.e., forany ) ifandonlyiftheyhavethesamemgfs(i.e., forany ). Proof Forafullygeneralproofofthis propositionsee,forexample,Feller(2008).Wejust giveaninformalproofforthespecialcaseinwhich and arediscreterandomvariablestakingonlyfinitelymanyvalues.The"onlyif" partistrivial.If and havethesamedistribution, thenThe "if"partisprovedasfollows.Denoteby and thesupportsof and andby and theirprobabilitymass functions.Denoteby theunionofthetwo supports:and by theelementsof . Themgfof canbewritten asBy thesametoken,themgfof canbewritten as:If and havethesamemgf,thenforany belongingtoaclosedneighborhoodof zeroandRearranging terms,we obtainThis canbetrueforany belongingtoaclosedneighborhoodofzeroonly iffor every . Itfollowsthatthattheprobabilitymassfunctionsof and areequal.Asaconsequence,alsotheirdistributionfunctionsareequal. Thispropositionisextremelyimportantandrelevantfromapractical viewpoint:inmanycaseswhereweneedtoprovethattwodistributionsare equal,itismucheasiertoproveequalityofthemomentgeneratingfunctions thantoproveequalityofthedistributionfunctions. Alsonotethatequalityofthedistributionfunctionscanbereplacedinthe propositionaboveby: equalityoftheprobabilitymassfunctions(if and arediscreterandom variables); equalityoftheprobabilitydensityfunctions(if and arecontinuous randomvariables). Moredetails Thefollowingsectionscontainmoredetailsaboutthemgf. Momentgeneratingfunctionofa lineartransformation Let bearandomvariablepossessingamgf . Definewhere aretwoconstantsand . Then,therandomvariable possessesamgf and Proof Bytheverydefinitionofmgf,we haveObviously, if isdefinedonaclosedinterval , then isdefinedontheinterval . Momentgeneratingfunctionofa sumofmutuallyindependentrandomvariables Let , ..., be mutuallyindependentrandomvariables. Let betheir sum: Then,themgfof istheproductofthemgfsof , ..., : Proof Thisiseasilyprovedbyusingthe definitionofmgfandthepropertiesofmutuallyindependent variables: Multivariategeneralization Themultivariategeneralizationofthemgfisdiscussedinthelectureonthe joint momentgeneratingfunction. Solvedexercises Somesolvedexercisesonmomentgeneratingfunctionscanbefoundbelow. Exercise1 Let beadiscreterandomvariablehavinga Bernoullidistribution. Itssupport isand itsprobabilitymass function iswhere isaconstant. Derivethemomentgeneratingfunctionof , ifitexists. Solution Bythedefinitionofmomentgenerating function,we haveObviously, themomentgeneratingfunctionexistsanditiswell-definedbecausetheabove expectedvalueexistsforany . Exercise2 Let bearandomvariablewithmomentgenerating function Derivethevarianceof . Solution Wecanusethefollowingformulafor computingthe variance:The expectedvalueof iscomputedbytakingthefirstderivativeofthemomentgenerating function:and evaluatingitat :The secondmomentof iscomputedbytakingthesecondderivativeofthemomentgenerating function:and evaluatingitat :Therefore, Exercise3 Arandomvariable issaidtohaveaChi-squaredistribution with degreesoffreedomifitsmomentgeneratingfunctionisdefinedforany anditisequal to Define where and aretwoindependentrandomvariableshavingChi-squaredistributionswith and degreesoffreedomrespectively. Provethat hasaChi-squaredistributionwith degreesoffreedom. Solution Themomentgeneratingfunctionsof and areThe momentgeneratingfunctionofasumofindependentrandomvariablesisjust theproductoftheirmomentgenerating functions:Therefore, isthemomentgeneratingfunctionofaChi-squarerandomvariablewith degreesoffreedom.Asaconsequence, hasaChi-squaredistributionwith degreesoffreedom. References Feller,W.(2008) Anintroduction toprobabilitytheoryanditsapplications,Volume2,Wiley. Pfeiffer,P.E.(1978) Conceptsof probabilitytheory,DoverPublications. Howtocite Pleaseciteas: Taboga,Marco(2021)."Momentgeneratingfunction",Lecturesonprobabilitytheoryandmathematicalstatistics.KindleDirectPublishing.Onlineappendix.https://www.statlect.com/fundamentals-of-probability/moment-generating-function. Thebooks Mostofthelearningmaterialsfoundonthiswebsitearenowavailableinatraditionaltextbookformat. ProbabilityandstatisticsMatrixalgebra Featuredpages Poissondistribution Uniformdistribution Bayesrule Deltamethod Permutations Combinations Explore Normaldistribution Meansquareconvergence Almostsureconvergence Mainsections Mathematicaltools Fundamentalsofprobability Probabilitydistributions Asymptotictheory Fundamentalsofstatistics Glossary About AboutStatlect Contacts Cookies,privacyandtermsofuse Glossaryentries TypeIerror Nullhypothesis TypeIIerror Precisionmatrix Posteriorprobability Estimator Share Toenhanceyourprivacy, weremovedthesocialbuttons, butdon'tforgettoshare.