Statsmodels 튜토리얼

Statsmodels 튜토리얼#

import statsmodels

하위모듈확인

help로 확인한다

help(statsmodels)

Help on package statsmodels:

NAME
    statsmodels

PACKAGE CONTENTS
    _version
    api
    base (package)
    compat (package)
    conftest
    datasets (package)
    discrete (package)
    distributions (package)
    duration (package)
    emplike (package)
    formula (package)
    gam (package)
    genmod (package)
    graphics (package)
    imputation (package)
    interface (package)
    iolib (package)
    miscmodels (package)
    multivariate (package)
    nonparametric (package)
    othermod (package)
    regression (package)
    robust (package)
    sandbox (package)
    src (package)
    stats (package)
    tests (package)
    tools (package)
    tsa (package)

FUNCTIONS
    test(extra_args=None, exit=False)
        Run the test suite
        
        Parameters
        ----------
        extra_args : list[str]
            List of argument to pass to pytest when running the test suite. The
            default is ['--tb=short', '--disable-pytest-warnings'].
        exit : bool
            Flag indicating whether the test runner should exist when finished.
        
        Returns
        -------
        int
            The status code from the test run if exit is False.

DATA
    debug_warnings = False

VERSION
    0.13.2

FILE
    /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/statsmodels/__init__.py

자주쓰는 모듈 구성#

statsmodels
│
├── 01 사후분석
│   │
│   └──stats
│       └── multicomp
│           ├── MultiComparison
│           │   └── allpairtest
│           └── pairwise_tukeyhsd
│
├── 02 시계열분석
│   │
│   ├── graphics.tsaplots
│   │   ├── plot_acf
│   │   └── plot_pacf
│   └── tsa
│       ├── arima_model
│       │   └── ARIMA
│       └── statesplace.sarimax
│           └── SARIMAX
│
├── 03 ANOVA (scipy모듈과 함께써야 모두 커버가능, 이분산 anova의 경우 pingouin모듈의 welch_anova를 사용)
│   │
│   ├── 다원분산분석 or 이원분산분석
│   └── 일원분산분석
│       └── stats.anova
│           └── anova_lm
│
└── 04 회귀분석
    │
    └── formula.api
        └── ols

세 집단 이상 검정 (independent)#

정규성(scipy.stats.shapiro), 등분산(scipy.stats.levene) 검정의 결과에 따라 시행하는 검정이 다르다
정규성 만족, 등분산 만족 : one-way anova
정규성 만족, 등분산 불만족 : welch’s anova
정규성 불만족 : kruskal-wallis H test

Question

x1,x2,x3의 변수들의 평균의 차이가 존재하는지 검정하라.
차이가 존재한다면 사후 분석까지 진행하라

import pandas as pd 
import matplotlib.pyplot as plt
df = pd.read_csv('https://raw.githubusercontent.com/Datamanim/datarepo/main/scipy/anova.csv')

data = df.x1
data2 = df.x2
data3 = df.x3

fig,ax = plt.subplots(1,2,figsize=(12,4))
ax[0].hist(data,alpha=0.3,bins=30,label='x1')
ax[0].hist(data2,alpha=0.3,bins=30,label='x2')
ax[0].hist(data3,alpha=0.3,bins=30,label='x3')
ax[0].legend()
labels = ['x1', 'x2', 'x3']
lst =[data,data2,data3]
ax[1].boxplot(lst, labels=labels) 
plt.show()

../../_images/6dadbcbd52ff1091352ad7f6c7b129777746ea0f8af9a09a09a654be9beda931.png

Question

x1,x2,x3의 변수들의 평균의 차이가 존재하는지 검정하라

import pandas as pd 
import matplotlib.pyplot as plt
df = pd.read_csv('https://raw.githubusercontent.com/Datamanim/datarepo/main/scipy/anova2.csv')

data = df.x1
data2 = df.x2
data3 = df.x3

fig,ax = plt.subplots(1,2,figsize=(12,4))
ax[0].hist(data,alpha=0.3,bins=30,label='x1')
ax[0].hist(data2,alpha=0.3,bins=30,label='x2')
ax[0].hist(data3,alpha=0.3,bins=30,label='x3')
ax[0].legend()
labels = ['x1', 'x2', 'x3']
lst =[data,data2,data3]
ax[1].boxplot(lst, labels=labels) 
plt.show()

../../_images/68cc8b2329b355ac19b0d701b08da40341c6ed6ea0eb560c4be4506b6cd6662c.png

Question

target 변수들에 의해 value값들의 평균의 차이가 존재하는지 검정하라

import pandas as pd 
import matplotlib.pyplot as plt
df = pd.read_csv('https://raw.githubusercontent.com/Datamanim/datarepo/main/scipy/anova8.csv')

data = df[df.target=='a'].value
data2 = df[df.target=='b'].value
data3 = df[df.target=='c'].value

fig,ax = plt.subplots(1,2,figsize=(12,4))
ax[0].hist(data,alpha=0.3,bins=30,label='a')
ax[0].hist(data2,alpha=0.3,bins=30,label='b')
ax[0].hist(data3,alpha=0.3,bins=30,label='c')
ax[0].legend()
labels = ['a', 'b', 'c']
lst =[data,data2,data3]
ax[1].boxplot(lst, labels=labels) 
plt.show()

../../_images/dfe071bf2f50e043734442965d81d8b9787eafc5309d0b6e719e5fa38ce00c97.png

Question

target 변수들에 의해 value값들의 평균의 차이가 존재하는지 검정하라

import pandas as pd 
import matplotlib.pyplot as plt
df = pd.read_csv('https://raw.githubusercontent.com/Datamanim/datarepo/main/scipy/anova10.csv')

data = df[df.target=='a'].value
data2 = df[df.target=='b'].value
data3 = df[df.target=='c'].value

fig,ax = plt.subplots(1,2,figsize=(12,4))
ax[0].hist(data,alpha=0.3,bins=30,label='a')
ax[0].hist(data2,alpha=0.3,bins=30,label='b')
ax[0].hist(data3,alpha=0.3,bins=30,label='c')
ax[0].legend()
labels = ['a', 'b', 'c']
lst =[data,data2,data3]
ax[1].boxplot(lst, labels=labels) 
plt.show()

../../_images/e5e88bf15291627df304f19682acd0374b7f069b3a98226a9fcc784988b4d2fd.png

Show code cell content Hide code cell content

from scipy.stats import shapiro

# 정규성 검정 -> 정규성 가짐
print(shapiro(data))
print(shapiro(data2))
print(shapiro(data3))

from scipy.stats import levene

# 등분산 만족하지 않음 -> welch test를 진행해야한다 (pingouin.welch_anova)
print(levene(data,data2,data3))
print()

# anova
from scipy.stats import f_oneway  ,kruskal
print(f_oneway(data, data2, data3))
print(kruskal(data, data2, data3))

import pingouin as pg
display(pg.welch_anova(dv='value', between='target', data=df))


# 비교를 위해 f_oneway 와 kruskal 모두 시행했다.
# kruskal의 경우 평균의 차이가 존재, f_oneway, welch(p-unc값)의 경우 차이가 존재하지 않는다

ShapiroResult(statistic=0.9835558533668518, pvalue=0.24877400696277618)
ShapiroResult(statistic=0.9923046231269836, pvalue=0.9479172825813293)
ShapiroResult(statistic=0.9523516893386841, pvalue=0.19537121057510376)
LeveneResult(statistic=4.530341210288782, pvalue=0.011923468290294031)

F_onewayResult(statistic=3.0058619004231084, pvalue=0.051771776524375666)
KruskalResult(statistic=7.273258848614091, pvalue=0.026340978769385875)

	Source	ddof1	ddof2	F	p-unc	np2
0	target	2	69.100989	2.244923	0.113608	0.029613

Statsmodels 튜토리얼

Contents

Statsmodels 튜토리얼#

자주쓰는 모듈 구성#

세 집단 이상 검정 (independent)#