-
Notifications
You must be signed in to change notification settings - Fork 116
Taking weighting seriously #487
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
gragusa
wants to merge
208
commits into
JuliaStats:master
Choose a base branch
from
gragusa:JuliaStats-master
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 1 commit
Commits
Show all changes
208 commits
Select commit
Hold shift + click to select a range
1754cbd
WIP
gragusa 1d778a5
WIP
gragusa 12121a3
WIP
gragusa 4363ba4
Taking weights seriously
gragusa ca702dc
WIP
gragusa e2b2d12
Taking weights seriously
gragusa bc8709a
Merge branch 'master' of https://github.com/JuliaStats/GLM.jl into Ju…
gragusa 84cd990
Add depwarn for passing wts with Vector
gragusa cbc329f
Cosmettic changes
gragusa 23d67f5
WIP
gragusa f4d90a9
Fix loglik for weighted models
gragusa 6b7d95c
Fix remaining issues
gragusa c236b82
Final commit
gragusa d4bd0c2
Merge branch 'master'
gragusa 8bdfb55
Fix merge
gragusa 3eb2ca4
Fix nulldeviance
gragusa 63c8358
Bypass crossmodelmatrix drom StatsAPI
gragusa e93a919
Delete momentmatrix.jl
gragusa 7bb0959
Delete scratch.jl
gragusa ded17a8
Delete settings.json
gragusa 3346774
AbstractWeights are required to be real
gragusa 7376e78
Update src/glmfit.jl
gragusa a738268
Apply suggestions from code review
gragusa c9459e7
Merge pull request #2 from JuliaStats/master
gragusa 6af3ca5
Throw error if GlmResp are not AbastractWeights
gragusa 0ded1d4
Addressing review comments
gragusa d923e48
Reexport aweights, pweights, fweights
gragusa 84f27d1
Fixed remaining issues with null loglikelihood
gragusa 8804dc1
Fix nullloglikelihood tests
gragusa 7f3aa36
Do not dispatch on Weights but use if
gragusa f67a8e0
Do not dispatch on Weights use if
gragusa 23a3e87
Fix inferred test
gragusa 5481284
Use if instead of dispatching on Weights
gragusa d12222e
Add doc for weights and fix output
gragusa a17e812
Fix docs failures
gragusa 58dec0c
Fix pweights stderror even for rank deficient des
gragusa a6f5c66
Add test for pweights stderror
gragusa 92ddb1e
Export UnitWeights
gragusa 0c61fff
Fix documentation
gragusa 8b0e8e1
Mkae cooksdistance work with rank deficient design
gragusa f609f06
Test cooksdistance with rank deficient design
gragusa 23f3d03
Fix CholeskyPivoted signature in docs
gragusa 2749b84
Make nancolidx v1.0 and v1.1 friendly
gragusa 82e472b
Fix signatures
gragusa 2d6aaed
Correct implementation of momentmatrix
gragusa dbc9ae9
Test moment matrix
gragusa e0d9cdf
Apply suggestions from code review
gragusa 46e8f92
Incorporate suggestions of reviewer
gragusa 6df401b
Deals with review comments
gragusa ca15eb8
Small fix
gragusa 0c18ae9
Small fix
gragusa 54d68d1
Apply suggestions from code review
gragusa 422a8cd
Merge branch 'master' into JuliaStats-master
gragusa d6d4e6b
Fix vcov dispatch for vcov
gragusa b457d74
Fix dispatch of _vcov
gragusa b087679
Revert changes
gragusa a44e137
Update src/glmfit.jl
gragusa 11db2c4
Fix weighted keyword in modelmatrix
gragusa b649d4f
perf in nulldeviance for unweighted models
gragusa 170148c
Merge branch 'JuliaStats-master' of github.com:gragusa/GLM.jl into Ju…
gragusa 29c43cb
Fixed std error for probability weights
gragusa 279e533
Getting there (& switch Analytics to Importance)
gragusa afb145e
.= instead of copy!
gragusa 2cead0a
Remove comments
gragusa a1ec49f
up
gragusa 97bf28d
Speedup cooksdistance
gragusa 9ce2d89
Revert back to AnalyticWeights
gragusa 9bddf63
Add extensive tests for AnalyticWeights
gragusa 3fe045a
Add extensive tests for AnalyticWeights
gragusa 852e307
Delete scratch.jl
gragusa d1ba3e5
Delete analytic_weights.jl
gragusa 831f280
Follow reviewer suggestions [Batch 1]
gragusa b00dc16
Follow reviewer's suggestions [Batch 2]
gragusa 0825324
probability weights vcov uses momentmatrix
gragusa 48d15fb
Fix ProbabilityWeights vcov and tests
gragusa 3338eab
Use leverage from StasAPI
gragusa c27c749
Merge branch 'master' into JuliaStats-master
gragusa 970e26e
Rebase against master
gragusa 8832e9d
Fix test
gragusa 9eb2390
Merge remote-tracking branch 'origin/master' into JuliaStats-master
gragusa 587c129
Test on 1.6
gragusa fa63a9a
Address reviwer comments
gragusa 807731a
Merge branch 'master' of github.com:JuliaStats/GLM.jl into JuliaStats…
gragusa 72996fc
Merge branch 'master' into JuliaStats-master
andreasnoack 1ee383a
Merge remote-tracking branch 'upstream/master' into JuliaStats-master
gragusa ba52ce9
Merge from origin
gragusa 5e790df
Fix broken test of dof_residual
gragusa 50c1a96
Fix testing issues
gragusa c4f7959
Fix docs
gragusa d2b5cb0
Added tests for ftest. They throw for pweights
gragusa cd165d7
Make ftest throw if a model weighted by pweights is passed
gragusa 606a419
Fix how loglikelihood throws for pweights weighted models
gragusa a1a1e10
Merge branch 'master' of github.com:JuliaStats/GLM.jl into JuliaStats…
gragusa 5d948de
Remove StatsPlots dependence.
gragusa 4fb18df
Fix weighting with :qr method.
gragusa 56d81ae
Add filter to jldoctest string
gragusa a2357cf
Fix problem with docstrings
gragusa 6068d2a
Update docs/src/index.md
gragusa 930a8cb
Remove trailing white spaces
gragusa 107d17d
Add mention of UnitWeights in the weights discussion
gragusa a003b10
Remove trailing white spaces
gragusa 1c06c7e
Change delbeta! signature
gragusa b41cce7
Add tests for dropcollinear=false
gragusa 2730277
Minor cosmethic changes
gragusa cdeb1a3
Add weighting information in COMMON_FIT_KWARGS_DOCS
gragusa 95d506e
Add test for leverage
gragusa f124589
[wip] work on leverage
gragusa cbdadbc
Use inverse
gragusa 2386ab9
Test leverage
gragusa 36326ff
Comment cookdistance
gragusa f26bc0e
Committed by mistake
gragusa 2bc2138
leverage returns a vec
gragusa 0569600
Fix cookdistance return type
gragusa dd1b4a8
Update docs/src/index.md
gragusa 1c5953d
Update docs/src/index.md
gragusa cd39578
Update src/glmfit.jl
gragusa 574ec69
Update src/linpred.jl
gragusa 60f43a8
Adress outstanding issues
gragusa cb7b0a0
Merge branch 'JuliaStats-master' of github.com:gragusa/GLM.jl into Ju…
gragusa a8a4a34
Merge branch 'master' into JuliaStats-master
gragusa 0f27b2f
Merge recent PRs
gragusa 9f778ff
The final frontier
gragusa cb0a518
Merge branch 'master' into JuliaStats-master
gragusa ce6d8fd
The true final frontier
gragusa 4d33245
Dealete dead code
gragusa 9d63624
Update test/runtests.jl
gragusa 214ce5e
Remove alien file
gragusa d6a5757
Merge branch 'JuliaStats-master' of github.com:gragusa/GLM.jl into Ju…
gragusa 59781e2
Update test/runtests.jl
gragusa 5023d30
Update src/lm.jl
gragusa 992dcaa
Update docs/src/examples.md
gragusa 4948b5a
Update src/linpred.jl
gragusa a005614
Merge branch 'JuliaStats-master' of github.com:gragusa/GLM.jl into Ju…
gragusa 841da03
General cleanup [ci skip]
gragusa 1d27c53
Add tests [ci skip]
gragusa 6c2f3e5
Fix sparse GLM
gragusa 4e9b765
Changes to make sparse reuse code
gragusa 89a1d7b
Add tests for sparse and other corner cases
gragusa e661db9
Do not access private API for weights [no ci]
gragusa 8db3d07
Re-add description of weights
gragusa e06acb5
Remove leftover [no ci]
gragusa c2a9103
Update probability_weights.jl
gragusa 36997f1
Delete .vscode directory [no ci]
gragusa c0263aa
Update glmfit.jl
gragusa 960f2fd
Update glmfit.jl [no ci]
gragusa 9db9bf4
Update analytic_weights.jl
gragusa 8a1bafb
Fix test_logs of warning in the binomial case
gragusa b977dd1
Update src/glmfit.jl
gragusa 3901150
FIx sparse (broken in master) and make loglikelihood error for analyt…
gragusa 6086c74
Improve Sparse case (vcov uses the dense code). Add tests for the spa…
gragusa e42d452
Update src/glmtools.jl [no ci]
gragusa 74ce605
Update ext/GLMSparseArraysExt.jl [no ci]
gragusa cfa10d2
Update src/glmfit.jl [no ci]
gragusa 02383cd
Add test for Sparse LM with probability weighted
gragusa 025d10c
Format code
gragusa 78db11d
Merge branch 'master' into JuliaStats-master
gragusa b7fec95
Implement cookdistance for glm. More testing.
gragusa 0a78140
Merge branch 'JuliaStats-master' of github.com:gragusa/GLM.jl into Ju…
gragusa 5a2af1c
Last commit.
gragusa 5dbd074
Update src/glmfit.jl [no ci]
gragusa 18d653a
Update src/lm.jl [no ci]
gragusa 0ff9ecb
Update src/lm.jl [no ci]
gragusa 328e898
Update src/lm.jl [no ci]
gragusa c06a174
Update src/glmfit.jl [no ci]
gragusa 3b07d1e
Update src/glmfit.jl [no ci]
gragusa 6efda93
Test residuals
gragusa ce6d39e
Merge branch 'JuliaStats-master' of github.com:gragusa/GLM.jl into Ju…
gragusa 92b6894
Correct how we test for wts
gragusa 4cef4c5
Add tests. Remove RDatasets from deps.
gragusa 88d4767
Fix negbin and add more tests
gragusa caa8c0f
Update src/glmfit.jl [no ci]
gragusa 3709d5e
Update src/glmfit.jl [no ci]
gragusa 18e8f4c
Update src/lm.jl [no ci]
gragusa 7ef083b
Update test/analytic_weights.jl [no ci]
gragusa 419c835
Update test/analytic_weights.jl [no ci]
gragusa 8507db1
Update src/lm.jl [no ci]
gragusa c6852a9
Update src/lm.jl [no ci]
gragusa a88b247
Update test/probability_weights.jl [no ci]
gragusa 6d66c88
Update src/glmfit.jl [no ci]
gragusa d4a9a51
Update src/linpred.jl [no ci]
gragusa 175b130
Update src/linpred.jl [no ci]
gragusa 740ddb7
Update src/linpred.jl [no ci]
gragusa 8f06f2a
Update src/linpred.jl [no ci]
gragusa 8668eb6
Update src/linpred.jl [no ci]
gragusa a1c1a99
Update src/linpred.jl [no ci]
gragusa 9523ae6
Update src/linpred.jl [no ci]
gragusa e7fbeac
Update src/linpred.jl [no ci]
gragusa c36c526
Update src/linpred.jl [no ci]
gragusa ce233b4
Update src/linpred.jl [no ci]
gragusa a3ff516
Update src/lm.jl [no ci]
gragusa 81299f1
Update src/lm.jl [no ci]
gragusa fa23a79
Update src/negbinfit.jl [no ci]
gragusa 28a950c
Update src/glmfit.jl [no ci]
gragusa 38ff9a2
Update src/glmfit.jl [no ci]
gragusa 064ad35
Update src/glmfit.jl [no ci]
gragusa 7635b10
Update src/glmfit.jl [no ci]
gragusa 53c2f8c
Update src/glmfit.jl [no ci]
gragusa 125a44f
Update src/glmfit.jl [no ci]
gragusa 5a5ae3d
Update src/glmfit.jl [no ci]
gragusa b839090
Fix comment in GlmResp
gragusa 9ecf9b5
Fix how wts are checked for empty size
gragusa efbcbf6
Fix formatting [noci]
gragusa 94d1044
Remove loglil_apweights_obs for Bernoulli and Binomial
gragusa ef6c86b
Fix formatting
gragusa 786dd99
Fix formatting
gragusa a3d68bc
Test linkinv with CauchitLink
gragusa aff48d6
Fix CauchitLink linkinv
gragusa 39300ab
Update src/lm.jl [no ci]
gragusa File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -87,10 +87,10 @@ | |
return -n / 2 * (log(2π * deviance(r) / n) + 1) | ||
end | ||
|
||
function loglikelihood(r::LmResp{T,<:AnalyticWeights}) where {T} | ||
N = length(r.y) | ||
n = sum(log, weights(r)) | ||
return (n - N * (log(2π * deviance(r) / N) + 1)) / 2 | ||
end | ||
|
||
function loglikelihood(r::LmResp{T,<:ProbabilityWeights}) where {T} | ||
|
@@ -169,7 +169,7 @@ | |
:fit) | ||
fweights(wts) | ||
else | ||
throw(ArgumentError("`wts` should be an `AbstractVector` coercible to `AbstractWeights`")) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Also worth testing. |
||
end | ||
end | ||
|
||
|
@@ -178,12 +178,12 @@ | |
dropcollinear::Bool=true, method::Symbol=:qr) | ||
# For backward compatibility accept wts as AbstractArray and coerce them to FrequencyWeights | ||
_wts = convert_weights(wts) | ||
if !(wts isa AbstractWeights && isempty(_wts)) | ||
|
||
if isempty(_wts) | ||
gragusa marked this conversation as resolved.
Show resolved
Hide resolved
|
||
Base.depwarn("Using `wts` of zero length for unweighted regression is deprecated in favor of " * | ||
"explicitly using `UnitWeights(length(y))`." * | ||
" Proceeding by coercing `wts` to UnitWeights of size $(length(y)).", | ||
:fit) | ||
_wts = uweights(length(y)) | ||
end | ||
|
||
if method === :cholesky | ||
|
@@ -202,7 +202,7 @@ | |
contrasts::AbstractDict{Symbol}=Dict{Symbol,Any}()) | ||
f, (y, X) = modelframe(f, data, contrasts, LinearModel) | ||
_wts = convert_weights(wts) | ||
_wts = !(wts isa AbstractWeights) && isempty(_wts) ? uweights(length(y)) : _wts | ||
_wts = isempty(_wts) ? uweights(length(y)) : _wts | ||
gragusa marked this conversation as resolved.
Show resolved
Hide resolved
gragusa marked this conversation as resolved.
Show resolved
Hide resolved
|
||
if method === :cholesky | ||
fit!(LinearModel(LmResp(y, _wts), cholpred(X, dropcollinear, _wts), f)) | ||
elseif method === :qr | ||
|
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Needs testing too.