基于Python fminunc 的替代方法
最近閑著沒(méi)事,想把coursera上斯坦福ML課程里面的練習(xí),用Python來(lái)實(shí)現(xiàn)一下,一是加深ML的基礎(chǔ),二是熟悉一下numpy,matplotlib,scipy這些庫(kù)。
在EX2中,優(yōu)化theta使用了matlab里面的fminunc函數(shù),不知道Python里面如何實(shí)現(xiàn)。搜索之后,發(fā)現(xiàn)stackflow上有人提到用scipy庫(kù)里面的minimize函數(shù)來(lái)替代。我嘗試直接調(diào)用我的costfunction和grad,程序報(bào)錯(cuò),提示(3,)和(100,1)dim維度不等,gradient vector不對(duì)之類(lèi)的,試了N多次后,終于發(fā)現(xiàn)問(wèn)題何在。。
首先來(lái)看看使用np.info(minimize)查看函數(shù)的介紹,傳入的參數(shù)有:
fun : callable
The objective function to be minimized.
``fun(x, *args) -> float``
where x is an 1-D array with shape (n,) and `args`
is a tuple of the fixed parameters needed to completely
specify the function.
x0 : ndarray, shape (n,)
Initial guess. Array of real elements of size (n,),
where 'n' is the number of independent variables.
args : tuple, optional
Extra arguments passed to the objective function and its
derivatives (`fun`, `jac` and `hess` functions).
method : str or callable, optional
Type of solver. Should be one of
- 'Nelder-Mead' :ref:`(see here) <optimize.minimize-neldermead>`
- 'Powell' :ref:`(see here) <optimize.minimize-powell>`
- 'CG' :ref:`(see here) <optimize.minimize-cg>`
- 'BFGS' :ref:`(see here) <optimize.minimize-bfgs>`
- 'Newton-CG' :ref:`(see here) <optimize.minimize-newtoncg>`
- 'L-BFGS-B' :ref:`(see here) <optimize.minimize-lbfgsb>`
- 'TNC' :ref:`(see here) <optimize.minimize-tnc>`
- 'COBYLA' :ref:`(see here) <optimize.minimize-cobyla>`
- 'SLSQP' :ref:`(see here) <optimize.minimize-slsqp>`
- 'trust-constr':ref:`(see here) <optimize.minimize-trustconstr>`
- 'dogleg' :ref:`(see here) <optimize.minimize-dogleg>`
- 'trust-ncg' :ref:`(see here) <optimize.minimize-trustncg>`
- 'trust-exact' :ref:`(see here) <optimize.minimize-trustexact>`
- 'trust-krylov' :ref:`(see here) <optimize.minimize-trustkrylov>`
- custom - a callable object (added in version 0.14.0),
see below for description.
If not given, chosen to be one of ``BFGS``, ``L-BFGS-B``, ``SLSQP``,
depending if the problem has constraints or bounds.
jac : {callable, '2-point', '3-point', 'cs', bool}, optional
Method for computing the gradient vector. Only for CG, BFGS,
Newton-CG, L-BFGS-B, TNC, SLSQP, dogleg, trust-ncg, trust-krylov,
trust-exact and trust-constr. If it is a callable, it should be a
function that returns the gradient vector:
``jac(x, *args) -> array_like, shape (n,)``
where x is an array with shape (n,) and `args` is a tuple with
the fixed parameters. Alternatively, the keywords
{'2-point', '3-point', 'cs'} select a finite
difference scheme for numerical estimation of the gradient. Options
'3-point' and 'cs' are available only to 'trust-constr'.
If `jac` is a Boolean and is True, `fun` is assumed to return the
gradient along with the objective function. If False, the gradient
will be estimated using '2-point' finite difference estimation.
需要注意的是fun關(guān)鍵詞參數(shù)里面的函數(shù),需要把優(yōu)化的theta放在第一個(gè)位置,X,y,放到后面。并且,theta在傳入的時(shí)候一定要是一個(gè)一維shape(n,)的數(shù)組,不然會(huì)出錯(cuò)。
然后jac是梯度,這里的有兩個(gè)地方要注意,第一個(gè)是傳入的theta依然要是一個(gè)一維shape(n,),第二個(gè)是返回的梯度也要是一個(gè)一維shape(n,)的數(shù)組。
總之,關(guān)鍵在于傳入的theta一定要是一個(gè)1D shape(n,)的,不然就不行。我之前為了方便已經(jīng)把theta塑造成了一個(gè)(n,1)的列向量,導(dǎo)致使用minimize時(shí)會(huì)報(bào)錯(cuò)。所以,學(xué)會(huì)用help看說(shuō)明可謂是相當(dāng)重要啊~
import numpy as np
import pandas as pd
import scipy.optimize as op
def LoadData(filename):
data=pd.read_csv(filename,header=None)
data=np.array(data)
return data
def ReshapeData(data):
m=np.size(data,0)
X=data[:,0:2]
Y=data[:,2]
Y=Y.reshape((m,1))
return X,Y
def InitData(X):
m,n=X.shape
initial_theta = np.zeros(n + 1)
VecOnes = np.ones((m, 1))
X = np.column_stack((VecOnes, X))
return X,initial_theta
def sigmoid(x):
z=1/(1+np.exp(-x))
return z
def costFunction(theta,X,Y):
m=X.shape[0]
J = (-np.dot(Y.T, np.log(sigmoid(X.dot(theta)))) - \
np.dot((1 - Y).T, np.log(1 - sigmoid(X.dot(theta))))) / m
return J
def gradient(theta,X,Y):
m,n=X.shape
theta=theta.reshape((n,1))
grad=np.dot(X.T,sigmoid(X.dot(theta))-Y)/m
return grad.flatten()
if __name__=='__main__':
data = LoadData('ex2data1csv.csv')
X, Y = ReshapeData(data)
X, initial_theta = InitData(X)
result = op.minimize(fun=costFunction, x0=initial_theta, args=(X, Y), method='TNC', jac=gradient)
print(result)
最后結(jié)果如下,符合MATLAB里面用fminunc優(yōu)化的結(jié)果(fminunc:cost:0.203,theta:-25.161,0.206,0.201)
fun: array([0.2034977]) jac: array([8.95038682e-09, 8.16149951e-08, 4.74505693e-07]) message: 'Local minimum reached (|pg| ~= 0)' nfev: 36 nit: 17 status: 0 success: True x: array([-25.16131858, 0.20623159, 0.20147149])
此外,由于知道cost在0.203左右,所以我用最笨的梯度下降試了一下,由于后面實(shí)在是太慢了,所以設(shè)置while J>0.21,循環(huán)了大概13W次。??梢?jiàn),使用集成好的優(yōu)化算法是多么重要。。。還有,在以前的理解中,如果一個(gè)學(xué)習(xí)速率不合適,J會(huì)一直發(fā)散,但是昨天的實(shí)驗(yàn)發(fā)現(xiàn),有的速率開(kāi)始會(huì)發(fā)散,后面還是會(huì)收斂。
以上這篇基于Python fminunc 的替代方法就是小編分享給大家的全部?jī)?nèi)容了,希望能給大家一個(gè)參考,也希望大家多多支持腳本之家。
相關(guān)文章
selenium+python自動(dòng)化測(cè)試之多窗口切換
這篇文章主要介紹了selenium+python自動(dòng)化測(cè)試之多窗口切換,小編覺(jué)得挺不錯(cuò)的,現(xiàn)在分享給大家,也給大家做個(gè)參考。一起跟隨小編過(guò)來(lái)看看吧2019-01-01
Python qrcode 生成一個(gè)二維碼的實(shí)例詳解
在本篇文章里小編給大家整理的是關(guān)于Python qrcode 生成一個(gè)二維碼的實(shí)例內(nèi)容,需要的朋友們可以學(xué)習(xí)參考下。2020-02-02
編寫(xiě)Python小程序來(lái)統(tǒng)計(jì)測(cè)試腳本的關(guān)鍵字
這篇文章主要介紹了編寫(xiě)Python小程序來(lái)統(tǒng)計(jì)測(cè)試腳本的關(guān)鍵字的方法,文中的實(shí)例不僅可以統(tǒng)計(jì)關(guān)鍵字?jǐn)?shù)量,還可以按主關(guān)鍵字來(lái)歸類(lèi),需要的朋友可以參考下2016-03-03
Python字符串逆序的實(shí)現(xiàn)方法【一題多解】
今天小編就為大家分享一篇關(guān)于Python字符串逆序的實(shí)現(xiàn)方法【一題多解】,小編覺(jué)得內(nèi)容挺不錯(cuò)的,現(xiàn)在分享給大家,具有很好的參考價(jià)值,需要的朋友一起跟隨小編來(lái)看看吧2019-02-02
python利用socket實(shí)現(xiàn)客戶(hù)端和服務(wù)端之間進(jìn)行通信
這篇文章主要介紹了python實(shí)現(xiàn)客戶(hù)端和服務(wù)端之間進(jìn)行通信,文章通過(guò)python利用socket展開(kāi)詳情介紹,具有一定的參考價(jià)值,需要的小伙伴可以參考一下2022-05-05
python虛擬機(jī)之描述器實(shí)現(xiàn)原理與源碼分析
在本篇文章當(dāng)中主要給大家介紹描述器背后的實(shí)現(xiàn)原理,通過(guò)分析?cpython對(duì)應(yīng)的源代碼了解與描述器相關(guān)的字節(jié)碼的指令,我們就可以真正了解到描述器背后的原理,需要的朋友可以參考下2023-05-05

