Linear Regression with regularization (lasso / ridge regression)


(Herger) #1

Hello,

I’m new to Math.Net Numerics and was browsing the functionalities in the doc and found the part “Regularization” in the linear regression topic missing.

Any plan for adding it right “out of the box” ?

Cheers
Best


(Herger) #3

Geezus,

I hope I won’t have to code an optimized Ridge Regression myself. My last numerical optimized implementation was a Restricted Boltzmann Machine in CUDA which took me a master thesis to achieve…


(Herger) #4

Just posting here after some boring google search about the question above, it seems that there is no free lib offering a good implementation of the regularized linear regression using the Normal Equations…

I’m probably gonna create a numerical version of it which should do the trick with GD.

I have not time to re-implement a dense matrix lib myself. None the less don’t hesitate to post any suggestion here if you have any.

Best.


(Christoph Rüegg) #5

Regularization has been on the backlog/wishlist for quite a while now. I don’t really have time to work on this area myself either - but if you happen to come up with an implementation I’d be very interested to integrate it into Math.NET Numerics.


(Herger) #6

Hi Christoph,

Thx for the reply, I indeed implemented this code for one of my project
(with obv better results than the plain R implementations) but it is for
the moment tightly coupled with dependent code. I will let you know if it
happens that I have enough time to generalize it properly in the future
when I will reuse this model for later projects.

Best.

Herger


(Yuriy Zaletskyy) #7

I implemented Ridge regression. I’d like to add Ridge regression to the toolbox but I’m a bit puzzled with a question where to start with mine implementation in order to integrate it into Math.Net Numerics naturally.