def FitObservedSSFR_Peaks(observable='groupcat', sfq='star-forming', Mrcut=18, position='central', Mfid=10.5): ''' Fit the (M*, SFR) positions of the SSFR distribution SF peak to a SFR(M*) linear parameterization. This is only for the SDSS group catalog. ''' if 'groupcat' not in observable: raise ValueError SSFRpeak = ObservedSSFR_Peaks(observable, sfq=sfq, Mrcut=Mrcut, position=position) SFRs = SSFRpeak['sfr'] Mass = SSFRpeak['mass'] # remove outliers if sfq == 'star-forming': # for the SDSS group catalog, the last mass bin has nearly no SF galaxies # so the estimate is inaccurate SFRs = SFRs[:-1] Mass = Mass[:-1] SFRs[0] = -0.4 SFRs[1] = -0.275 p0 = [0.6, 0.] # guess elif sfq == 'quiescent': SFRs[0] = -1.925 SFRs[-1] = -1.475 p0 = [-0.4, 0.] print SFRs fa = {'x': Mass - Mfid, 'y': SFRs} bestfit = mpfit.mpfit(util.mpfit_line, p0, functkw=fa, quiet=True) return bestfit.params
def onedgaussfit(xax,data,err=None,params=[0,1,0,1],fixed=[False,False,False,False],limitedmin=[False,False,False,True], limitedmax=[False,False,False,False],minpars=[0,0,0,0],maxpars=[0,0,0,0], quiet=True,shh=True): """ Inputs: xax - x axis data - y axis err - error corresponding to data params - Fit parameters: Height of background, Amplitude, Shift, Width fixed - Is parameter fixed? limitedmin/minpars - set lower limits on each parameter (default: width>0) limitedmax/maxpars - set upper limits on each parameter quiet - should MPFIT output each iteration? shh - output final parameters? Returns: Fit parameters Model Fit errors chi2 """ def mpfitfun(x,y,err): if err == None: def f(p,fjac=None): return [0,(y-onedgaussian(x,*p))] else: def f(p,fjac=None): return [0,(y-onedgaussian(x,*p))/err] return f if xax == None: xax = numpy.arange(len(data)) parinfo = [ {'n':0,'value':params[0],'limits':[minpars[0],maxpars[0]],'limited':[limitedmin[0],limitedmax[0]],'fixed':fixed[0],'parname':"HEIGHT",'error':0} , {'n':1,'value':params[1],'limits':[minpars[1],maxpars[1]],'limited':[limitedmin[1],limitedmax[1]],'fixed':fixed[1],'parname':"AMPLITUDE",'error':0}, {'n':2,'value':params[2],'limits':[minpars[2],maxpars[2]],'limited':[limitedmin[2],limitedmax[2]],'fixed':fixed[2],'parname':"SHIFT",'error':0}, {'n':3,'value':params[3],'limits':[minpars[3],maxpars[3]],'limited':[limitedmin[3],limitedmax[3]],'fixed':fixed[3],'parname':"WIDTH",'error':0}] mp = mpfit(mpfitfun(xax,data,err),parinfo=parinfo,quiet=quiet) mpp = mp.params mpperr = mp.perror chi2 = mp.fnorm if mp.status == 0: raise Exception(mp.errmsg) if not shh: for i,p in enumerate(mpp): parinfo[i]['value'] = p print parinfo[i]['parname'],p," +/- ",mpperr[i] print "Chi2: ",mp.fnorm," Reduced Chi2: ",mp.fnorm/len(data)," DOF:",len(data)-len(mpp) return mpp,onedgaussian(xax,*mpp),mpperr,chi2
def FitObservedSFMS(observable, Mfid=10.5, slope=None, mass_range=None, **kwargs): ''' Fit the observed SFMS relation with a linear function. Parameters ---------- observable : str String that specifies the observables. SDSS/PRIMUS or SDSS Group Catalog Mfid : float Fiducial mass where the SFMS is pivoted around. A( M* - M_fid) + B. ''' # read in the observed SFMS mass, muSFR, sigSFR, ngal = ObservedSFMS(observable, **kwargs) if mass_range is not None: m_min, m_max = mass_range mlim = np.where((mass > m_min) & (mass < m_max)) mass = mass[mlim] muSFR = muSFR[mlim] sigSFR = sigSFR[mlim] ngal = ngal[mlim] # amplitude of SFMS depends on z so the initial guess # depends on z if observable == 'groupcat': guess_z = 0.05 elif observable == 'sdssprimus': guess_z = kwargs['redshift'] guess_z *= 0.7 if slope is None: p0 = [0.6, guess_z] fa = {'x': mass - Mfid, 'y': muSFR, 'err': sigSFR} bestfit = mpfit.mpfit(util.mpfit_line, p0, functkw=fa, quiet=True) else: p0 = [guess_z] fa = {'x': mass - Mfid, 'y': muSFR, 'err': sigSFR, 'slope': slope} bestfit = mpfit.mpfit(util.mpfit_line_fixedslope, p0, functkw=fa, quiet=True) return bestfit.params
def fitGauss(xdata,ydata,yerr,flatLine=False): nBins=100 amplitude = .5*np.max(ydata) x_offset = xdata[np.argmax(ydata)] sigma = (np.max(xdata)-np.min(xdata))/10. y_offset = 3. fixed = [False]*4 if flatLine == True: amplitude = 0 fixed[0:3] = [True]*3 params=[sigma, x_offset, amplitude, y_offset] # First guess at fit params errs = yerr errs[np.where(errs == 0.)] = 1. quiet = True parinfo = [ {'n':0,'value':params[0],'limits':[.0001, .1], 'limited':[True,True],'fixed':fixed[0],'parname':"Sigma",'error':0}, {'n':1,'value':params[1],'limits':[x_offset-sigma*3, x_offset+sigma*3],'limited':[True,True],'fixed':fixed[1],'parname':"x offset",'error':0}, {'n':2,'value':params[2],'limits':[.2*amplitude, 3.*amplitude],'limited':[True,True],'fixed':fixed[2],'parname':"Amplitude",'error':0}, {'n':3,'value':params[3],'limited':[False,False],'fixed':fixed[3],'parname':"y_offset",'error':0}] fa = {'x':xdata,'y':ydata,'err':yerr} m = mpfit.mpfit(gaussian, functkw=fa, parinfo=parinfo, maxiter=1000, quiet=quiet) if m.status <= 0: print m.status, m.errmsg mpp = m.params #The fit params mpperr = m.perror for k,p in enumerate(mpp): parinfo[k]['value'] = p parinfo[k]['error'] = mpperr[k] #print parinfo[k]['parname'],p," +/- ",mpperr[j] if k==0: sigma = p if k==1: x_offset = p if k==2: amplitude = p if k==3: y_offset = p fineXdata = np.linspace(np.min(xdata),np.max(xdata),100.) gaussfit = y_offset + amplitude * np.exp( - (( xdata - x_offset)**2) / ( 2. * (sigma**2))) fineGaussFit = y_offset + amplitude * np.exp( - (( fineXdata - x_offset)**2) / ( 2. * (sigma**2))) resolution = np.abs(x_offset/(2.355*sigma)) return {'gaussfit':gaussfit,'resolution':resolution,'sigma':sigma,'x_offset':x_offset,'amplitude':amplitude,'y_offset':y_offset,'fineXdata':fineXdata,'fineGaussFit':fineGaussFit,'parinfo':parinfo}
def Lee2015_SFMS_zslope(): ''' Calculate the sloep of the redshift dependence of the Lee et al. (2015) SFMS parameterizations Lee et al. (2015) parameterized SFMS: log SFR(M*, z) = S0(z) - log( 1 + (M*/M0)^-gamma) S0(z) quantifies the redshift dependence of the SFMS. Lee et al. (2015) fits S0 for each redshift bin. I fit: S0(z) = A_z * ( z - 0.0502) + C returns [A_z, C] ''' z_mid = np.array([0.36, 0.55, 0.70, 0.85, 0.99, 1.19]) S0 = np.array([0.80, 0.99, 1.23, 1.35, 1.53, 1.72]) S0_err = np.array([0.019, 0.015, 0.016, 0.014, 0.017, 0.024]) p0 = [1.5, 0.0] fa = {'x': z_mid-0.05, 'y': S0, 'err': S0_err} bestfit = mpfit.mpfit(util.mpfit_line, p0, functkw=fa, quiet=True) return bestfit.params
def get_bestfit_sfms_groupcat(Mrcut=18, fid_mass=10.5, clobber=False): ''' Calculate the linear bestfit parameters for the StarForming Main Sequence of the SDSS Group Catalog specified by Mrcut or SDSS+PRIMUS envcount catalog from qf_env project. Fitting is done using MPFit ---------------------------------------------------------------- Parameters ---------------------------------------------------------------- Mrcut : absolute magntiude cut that specifies the group catalog clobber : Rewrite if True ---------------------------------------------------------------- Notes ---------------------------------------------------------------- * Bestfit values are accessed from file, unless it doesn't exist or clobber == True ''' bestfit_file = ''.join([ 'dat/central_quenching/sf_ms/' 'sf_ms_fit_starforming_groupcat_', str(Mrcut), '.hdf5' ]) if Mrcut == 18: z_med = 0.03 elif Mrcut == 19: z_med = 0.05 elif Mrcut == 20: z_med = 0.08 if not os.path.isfile(bestfit_file) or clobber: print 'Writing ' print bestfit_file avg_sfrs, var_sfrs, ngal = get_sfr_mstar_z_groupcat(np.arange( 9.0, 11.5, 0.25), Mrcut=Mrcut) enough_gal = np.where(ngal > 100) masses = np.arange(9.0, 11.5, 0.25)[enough_gal] avg_sfrs = np.array(avg_sfrs)[enough_gal] var_sfrs = np.array(var_sfrs)[enough_gal] p0 = [0.5607, 0.0775917] # guess fa = { 'x': np.array(masses) - fid_mass, 'y': np.array(avg_sfrs), 'err': np.array(var_sfrs) } bestfit = mpfit.mpfit(mpfit_line, p0, functkw=fa) # save data to h5py file f = h5py.File(bestfit_file, 'w') grp = f.create_group('slope_yint') grp.attrs['fid_mass'] = fid_mass grp.create_dataset('zmid', data=[z_med]) grp.create_dataset('slope', data=[0.65]) grp.create_dataset('yint', data=[0.0]) #grp.create_dataset('slope', data=[bestfit.params[0]]) #grp.create_dataset('yint', data=[bestfit.params[1]]) f.close() #return [0.1, bestfit.params[0], bestfit.params[1]] return [z_med, 0.65, 0.0] else: f = h5py.File(bestfit_file, 'r') zmids = f['slope_yint/zmid'][:] slopes = f['slope_yint/slope'][:] yints = f['slope_yint/yint'][:] f.close() return [zmids, slopes, yints]
def get_bestfit_sfms_envcount(fid_mass=10.5, clobber=False): ''' Calculate linear bestfit parameters for SF-MS fits for redshift bins with delta z = 0.2. The slope and y-int are fit for z ~ 0.1. While only y-int is fit for z > 0.2 ''' bestfit_file = ''.join( ['dat/central_quenching/sf_ms/' 'sf_ms_fit_starforming_envcount.hdf5']) if not os.path.isfile(bestfit_file) or clobber: print 'Writing ' print bestfit_file # first calculate average SFR, sigma_sfr as a function of mass # for SDSS redshift bin of the envcount catalog. Afterwards # fit slope and yint to the SF-MS. Notes that mass range is higher # than later in the function. avg_sfrs, sig_sfrs, ngal = get_sfr_mstar_z_envcount( np.arange(9.0, 11.5, 0.25), [0.1 for i in xrange(len(np.arange(9.0, 11.5, 0.25)))]) enough_gal = np.where(np.array(avg_sfrs) != -10.) masses = np.arange(9.0, 11.5, 0.25)[enough_gal] avg_sfrs = np.array(avg_sfrs)[enough_gal] sig_sfrs = np.array(sig_sfrs)[enough_gal] p0 = [0.0775917] # guess fa = { 'x': np.array(masses) - fid_mass, 'y': np.array(avg_sfrs), 'err': np.array(sig_sfrs) } bestfit = mpfit.mpfit(mpfit_line_fixedslope, p0, functkw=fa) sdsszbin_slope = 0.65 #bestfit.params[0] sdsszbin_yint = 0.2 #bestfit.params[0] # use bestfit slope of low z SDSS bin to fit fixed slope # lines to the rest of the redshift bins zmids, slopes, yints = [], [], [] zbins = [0.3, 0.5, 0.7, 0.9] for zbin in zbins: avg_sfrs, sig_sfrs, ngal = get_sfr_mstar_z_envcount( np.arange(9.0, 11.5, 0.25), [zbin for i in xrange(len(np.arange(9.0, 11.5, 0.25)))]) enough_gal = np.where(np.array(avg_sfrs) > -10.) masses = np.arange(9.0, 11.5, 0.25)[enough_gal] avg_sfrs = np.array(avg_sfrs)[enough_gal] sig_sfrs = np.array(sig_sfrs)[enough_gal] p0 = [0.5] # bad guess fa = { 'slope': sdsszbin_slope, 'x': np.array(masses) - fid_mass, 'y': np.array(avg_sfrs), 'err': np.array(sig_sfrs) } bestfit = mpfit.mpfit(mpfit_line_fixedslope, p0, functkw=fa, quiet=1) zmids.append(zbin) slopes.append(sdsszbin_slope) yints.append(bestfit.params[0]) zmids.insert(0, 0.1) slopes.insert(0, sdsszbin_slope) yints.insert(0, sdsszbin_yint) f = h5py.File(bestfit_file, 'w') grp = f.create_group('slope_yint') grp.attrs['fid_mass'] = fid_mass grp.create_dataset('zmid', data=zmids) grp.create_dataset('slope', data=slopes) grp.create_dataset('yint', data=yints) f.close() return [zmids, slopes, yints] else: f = h5py.File(bestfit_file, 'r') zmids = f['slope_yint/zmid'][:] slopes = f['slope_yint/slope'][:] yints = f['slope_yint/yint'][:] f.close() return [zmids, slopes, yints]
def multigaussfit(xax,data,ngauss=1,err=None,params=[1,0,1],fixed=[False,False,False],limitedmin=[False,False,True], limitedmax=[False,False,False],minpars=[0,0,0],maxpars=[0,0,0], quiet=True,shh=True): """ An improvement on onedgaussfit. Lets you fit multiple gaussians. Inputs: xax - x axis data - y axis ngauss - How many gaussians to fit? Default 1 (this could supercede onedgaussfit) err - error corresponding to data These parameters need to have length = 3*ngauss. If ngauss > 1 and length = 3, they will be replicated ngauss times, otherwise they will be reset to defaults: params - Fit parameters: [amplitude, offset, width] * ngauss If len(params) % 3 == 0, ngauss will be set to len(params) / 3 fixed - Is parameter fixed? limitedmin/minpars - set lower limits on each parameter (default: width>0) limitedmax/maxpars - set upper limits on each parameter quiet - should MPFIT output each iteration? shh - output final parameters? Returns: Fit parameters Model Fit errors chi2 """ if len(params) != ngauss and (len(params) / 3) > ngauss: ngauss = len(params) / 3 # make sure all various things are the right length; if they're not, fix them using the defaults for parlist in (params,fixed,limitedmin,limitedmax,minpars,maxpars): if len(parlist) != 3*ngauss: # if you leave the defaults, or enter something that can be multiplied by 3 to get to the # right number of gaussians, it will just replicate if len(parlist) == 3: parlist *= ngauss elif parlist==params: parlist[:] = [1,0,1] * ngauss elif parlist==fixed or parlist==limitedmax: parlist[:] = [False,False,False] * ngauss elif parlist==limitedmin: parlist[:] = [False,False,True] * ngauss elif parlist==minpars or parlist==maxpars: parlist[:] = [0,0,0] * ngauss def mpfitfun(x,y,err): if err == None: def f(p,fjac=None): return [0,(y-n_gaussian(pars=p)(x))] else: def f(p,fjac=None): return [0,(y-n_gaussian(pars=p)(x))/err] return f if xax == None: xax = numpy.arange(len(data)) parnames = {0:"SHIFT",1:"WIDTH",2:"AMPLITUDE"} parinfo = [ {'n':ii,'value':params[ii],'limits':[minpars[ii],maxpars[ii]],'limited':[limitedmin[ii],limitedmax[ii]],'fixed':fixed[ii],'parname':parnames[ii/3]+str(ii/3),'error':ii} for ii in xrange(len(params)) ] mp = mpfit(mpfitfun(xax,data,err),parinfo=parinfo,quiet=quiet) mpp = mp.params mpperr = mp.perror chi2 = mp.fnorm if mp.status == 0: raise Exception(mp.errmsg) if not shh: for i,p in enumerate(mpp): parinfo[i]['value'] = p print parinfo[i]['parname'],p," +/- ",mpperr[i] print "Chi2: ",mp.fnorm," Reduced Chi2: ",mp.fnorm/len(data)," DOF:",len(data)-len(mpp) return mpp,n_gaussian(pars=mpp)(xax),mpperr,chi2
def gaussfit(data,err=None,params=[],autoderiv=1,return_all=0,circle=0, fixed=numpy.repeat(False,7),limitedmin=[False,False,False,False,True,True,True], limitedmax=[False,False,False,False,False,False,True], usemoment=numpy.array([],dtype='bool'), minpars=numpy.repeat(0,7),maxpars=[0,0,0,0,0,0,360], rotate=1,vheight=1,quiet=True,returnmp=False, returnfitimage=False,**kwargs): """ Gaussian fitter with the ability to fit a variety of different forms of 2-dimensional gaussian. Input Parameters: data - 2-dimensional data array err=None - error array with same size as data array params=[] - initial input parameters for Gaussian function. (height, amplitude, x, y, width_x, width_y, rota) if not input, these will be determined from the moments of the system, assuming no rotation autoderiv=1 - use the autoderiv provided in the lmder.f function (the alternative is to us an analytic derivative with lmdif.f: this method is less robust) return_all=0 - Default is to return only the Gaussian parameters. 1 - fit params, fit error returnfitimage - returns (best fit params,best fit image) returnmp - returns the full mpfit struct circle=0 - default is an elliptical gaussian (different x, y widths), but can reduce the input by one parameter if it's a circular gaussian rotate=1 - default allows rotation of the gaussian ellipse. Can remove last parameter by setting rotate=0. numpy.expects angle in DEGREES vheight=1 - default allows a variable height-above-zero, i.e. an additive constant for the Gaussian function. Can remove first parameter by setting this to 0 usemoment - can choose which parameters to use a moment estimation for. Other parameters will be taken from params. Needs to be a boolean array. Output: Default output is a set of Gaussian parameters with the same shape as the input parameters Can also output the covariance matrix, 'infodict' that contains a lot more detail about the fit (see scipy.optimize.leastsq), and a message from leastsq telling what the exit status of the fitting routine was Warning: Does NOT necessarily output a rotation angle between 0 and 360 degrees. """ usemoment=numpy.array(usemoment,dtype='bool') params=numpy.array(params,dtype='float') if usemoment.any() and len(params)==len(usemoment): moment = numpy.array(moments(data,circle,rotate,vheight,**kwargs),dtype='float') params[usemoment] = moment[usemoment] elif params == [] or len(params)==0: params = (moments(data,circle,rotate,vheight,**kwargs)) if vheight==0: vheight=1 params = numpy.concatenate([[0],params]) fixed[0] = 1 # mpfit will fail if it is given a start parameter outside the allowed range: for i in xrange(len(params)): if params[i] > maxpars[i] and limitedmax[i]: params[i] = maxpars[i] if params[i] < minpars[i] and limitedmin[i]: params[i] = minpars[i] if err == None: errorfunction = lambda p: numpy.ravel((twodgaussian(p,circle,rotate,vheight)\ (*numpy.indices(data.shape)) - data)) else: errorfunction = lambda p: numpy.ravel((twodgaussian(p,circle,rotate,vheight)\ (*numpy.indices(data.shape)) - data)/err) def mpfitfun(data,err): if err == None: def f(p,fjac=None): return [0,numpy.ravel(data-twodgaussian(p,circle,rotate,vheight)\ (*numpy.indices(data.shape)))] else: def f(p,fjac=None): return [0,numpy.ravel((data-twodgaussian(p,circle,rotate,vheight)\ (*numpy.indices(data.shape)))/err)] return f parinfo = [ {'n':1,'value':params[1],'limits':[minpars[1],maxpars[1]],'limited':[limitedmin[1],limitedmax[1]],'fixed':fixed[1],'parname':"AMPLITUDE",'error':0}, {'n':2,'value':params[2],'limits':[minpars[2],maxpars[2]],'limited':[limitedmin[2],limitedmax[2]],'fixed':fixed[2],'parname':"XSHIFT",'error':0}, {'n':3,'value':params[3],'limits':[minpars[3],maxpars[3]],'limited':[limitedmin[3],limitedmax[3]],'fixed':fixed[3],'parname':"YSHIFT",'error':0}, {'n':4,'value':params[4],'limits':[minpars[4],maxpars[4]],'limited':[limitedmin[4],limitedmax[4]],'fixed':fixed[4],'parname':"XWIDTH",'error':0} ] if vheight == 1: parinfo.insert(0,{'n':0,'value':params[0],'limits':[minpars[0],maxpars[0]],'limited':[limitedmin[0],limitedmax[0]],'fixed':fixed[0],'parname':"HEIGHT",'error':0}) if circle == 0: parinfo.append({'n':5,'value':params[5],'limits':[minpars[5],maxpars[5]],'limited':[limitedmin[5],limitedmax[5]],'fixed':fixed[5],'parname':"YWIDTH",'error':0}) if rotate == 1: parinfo.append({'n':6,'value':params[6],'limits':[minpars[6],maxpars[6]],'limited':[limitedmin[6],limitedmax[6]],'fixed':fixed[6],'parname':"ROTATION",'error':0}) if autoderiv == 0: # the analytic derivative, while not terribly difficult, is less # efficient and useful. I only bothered putting it here because I was # instructed to do so for a class project - please ask if you would # like this feature implemented raise ValueError("I'm sorry, I haven't implemented this feature yet.") else: # p, cov, infodict, errmsg, success = optimize.leastsq(errorfunction,\ # params, full_output=1) mp = mpfit(mpfitfun(data,err),parinfo=parinfo,quiet=quiet,maxiter=1000) print mp.status, if mp.status <= 0: print mp.errmsg if returnmp: returns = (mp) elif return_all == 0: returns = mp.params elif return_all == 1: returns = mp.params,mp.perror if returnfitimage: fitimage = twodgaussian(mp.params,circle,rotate,vheight)(*numpy.indices(data.shape)) returns = (returns,fitimage) return returns
def FqCen_bestfit(clobber=False): ''' We parameterize the central galaxy quiescent fraction as fQ(M*, z) = fQ(M*, z = 0.) * (1+z)^(alpha(M*)). Here we fit for alpha log fQ(M*,z) - log fQ(M*, z=0) = alpha(M*) * log(1+z) ''' # mass binnning we impose m_low = np.array([9.5, 10., 10.5, 11., 11.5]) m_high = np.array([10., 10.5, 11., 11.5, 12.0]) m_mid = 0.5 * (m_low + m_high) # SDSS fq_file = ''.join(['dat/observations/cosmos_fq/', 'fcen_red_sdss_scatter.dat']) m_sdss, fqcen_sdss, N_sdss = np.loadtxt(fq_file, unpack=True, usecols=[0,1,2]) fqcen_sdss_rebin = [] for im, m_mid_i in enumerate(m_mid): sdss_mbin = np.where( (m_sdss >= m_low[im]) & (m_sdss < m_high[im])) fqcen_sdss_rebin.append( np.sum(fqcen_sdss[sdss_mbin] * N_sdss[sdss_mbin].astype('float'))/np.sum(N_sdss[sdss_mbin].astype('float')) ) fqcen_sdss_rebin = np.array(fqcen_sdss_rebin) fqcen_cosmos_rebin = [] fqcen_low_cosmos_rebin = [] fqcen_high_cosmos_rebin = [] for iz, z in enumerate([0.36, 0.66, 0.88]): fq_file = ''.join(['dat/observations/cosmos_fq/', 'stats_z', str(iz+1), '.fq_cen']) m_cosmos, fqcen_cosmos, fqcen_cosmos_low, fqcen_cosmos_high = np.loadtxt(fq_file, unpack=True, usecols=[0,1,2,3]) m_cosmos = np.log10(m_cosmos) fqcen_interp = interpolate.interp1d(m_cosmos, fqcen_cosmos) fqcen_low_interp = interpolate.interp1d(m_cosmos, fqcen_cosmos_low) fqcen_high_interp = interpolate.interp1d(m_cosmos, fqcen_cosmos_high) fqcen_cosmos_rebin.append(fqcen_interp(m_mid)) fqcen_low_cosmos_rebin.append(fqcen_low_interp(m_mid)) fqcen_high_cosmos_rebin.append(fqcen_high_interp(m_mid)) norm_fqcen_cosmos_rebin = np.array([ fqcen_cosmos_rebin[ii]/fqcen_sdss_rebin for ii in range(len(fqcen_cosmos_rebin)) ]) norm_fqcen_low_cosmos_rebin = np.array([ fqcen_low_cosmos_rebin[ii]/fqcen_sdss_rebin for ii in range(len(fqcen_cosmos_rebin)) ]) norm_fqcen_high_cosmos_rebin = np.array([ fqcen_high_cosmos_rebin[ii]/fqcen_sdss_rebin for ii in range(len(fqcen_cosmos_rebin)) ]) z_arr = np.array([0.0, 0.36, 0.66, 0.88]) alpha_m = [] for im, mm in enumerate(m_mid): # fit the redshift evolution with a power law for each mass bin # using mpfit with asymmetric errors from Tinker et al. (2013)'s # errors p0 = [-2.] fa = { 'x': z_arr, 'y': np.log10(np.array([1.] + list(norm_fqcen_cosmos_rebin[:,im]))), 'y_low': np.log10(np.array([.999] + list(norm_fqcen_low_cosmos_rebin[:,im]))), 'y_high': np.log10(np.array([1.001] + list(norm_fqcen_high_cosmos_rebin[:,im]))) } bestfit = mpfit.mpfit(mpfit_z_powerlaw, p0, functkw=fa, quiet=True) alpha_m.append(bestfit.params[0]) output_file = ''.join([os.path.dirname(os.path.realpath(__file__)).split('CenQue')[0], 'dat/fqcen_alphaM.dat']) if not os.path.isfile(output_file) or clobber: np.savetxt(output_file, np.array([m_mid, alpha_m]).T, fmt=['%10.5f', '%10.5f']) return [m_mid, alpha_m]
def fitData2D(data,errs,parameter_guess,parameter_lowerlimit,parameter_upperlimit,model,parameter_ties=None,verbose=False): """ Runs mpfit.py. Gets all the parameters in the right format, calls mpfit, parses output. This version scales your guess to 1.0, then fits for a percent difference (unitless). Then it scales back (puts the units back) and returns the fitted parameters. Inputs: data - 2d array of data (0 for dead pixel) errs - 2d array of errors (np.inf for dead pixel, 1 for pixel with no counts) parameter_guess - Best guess for parameters parameter_lowerlimit - Strict lower limit in parameter space parameter_upperlimit - Strict upper limit in parameter space - None means no limit - If lower and upper limits are the same then the parameter is fixed model - [String] model used to fit data. Must be contained in model_list. See /util/fitFunctions.py parameter_ties - array of length parameter_guess - [0,0,1,0,1,2,0,2,0] means parameters 3 and 5 should be tied and parameters 6 and 8 should be tied. - don't tie parameters with flags <=0 verbose - Show runtime comments **kwargs - keyword parameters for model Outputs: parameter_fit - array of parameters for best fit redchi2gauss2 - the reduced chi^2 of the fit mpperr - array of errors on fit parameters """ #Create list of parameters in format that mpfit likes parinfo = [] for k in range(len(parameter_guess)): lowLimit=True highLimit=True if parameter_lowerlimit[k]==None: lowLimit=False if parameter_upperlimit[k]==None: highLimit=False fix_guess = False if parameter_lowerlimit[k]==parameter_upperlimit[k] and parameter_lowerlimit[k]!=None: fix_guess=True #p_ll = None if parameter_lowerlimit[k]==None else 1.0*parameter_lowerlimit[k]/parameter_guess[k] #p_ul = None if parameter_upperlimit[k]==None else 1.0*parameter_upperlimit[k]/parameter_guess[k] #par = {'n':k,'value':1.,'limits':[p_ll, p_ul],'limited':[lowLimit,highLimit],'fixed':fix_guess,'error':0} par = {'n':k,'value':parameter_guess[k],'limits':[parameter_lowerlimit[k],parameter_upperlimit[k]],'limited':[lowLimit,highLimit],'fixed':fix_guess,'error':0} parinfo.append(par) #Tie some parameters together if specified if parameter_ties!=None and len(parameter_ties)==len(parameter_guess): for p_flag in np.unique(np.asarray(parameter_ties)[np.where(np.asarray(parameter_ties)>0)]): p_to_tie = np.where(np.asarray(parameter_ties)==p_flag)[0] if len(p_to_tie)>1: for p in p_to_tie[1:]: parinfo[p]['tied'] = 'p['+str(p_to_tie[0])+']' #put parameters for fitting function in dictionary fa = {'data':data,'err':errs} quiet=True #Run mpfit, catch annoying warnings with warnings.catch_warnings(): warnings.simplefilter("ignore") #m = mpfit.mpfit(model_list[model], functkw=fa, ftol=1.e-15, xtol=1.e-20, parinfo=parinfo, maxiter=2000, quiet=quiet) #m = mpfit(model_list[model](parameter_guess), functkw=fa, parinfo=parinfo, maxiter=1000, quiet=quiet) m = mpfit(model_list[model](np.ones(len(parameter_guess))), functkw=fa, parinfo=parinfo, maxiter=1000, quiet=quiet) mpp = m.params #The fit params #mpperr = (np.asarray(m.perror)*np.asarray(parameter_guess)).tolist() mpperr = m.perror chi2gauss = m.fnorm dof = np.sum(np.isfinite(errs)) #degrees of freedom redchi2gauss2 = 1.0*chi2gauss/dof if verbose: print "Status: "+str(m.status)+" after "+str(m.niter)+" iterations" print "mpperr: "+str(mpperr) print "reduced Chi^2: "+str(redchi2gauss2) #print "fnorm: "+str(m.fnorm) if mpperr==None: print m.errmsg #Parse out fitted parameters parameter_fit=np.copy(np.asarray(parameter_guess)) for k,p in enumerate(mpp): parinfo[k]['value'] = p #parameter_fit[k]=p*parameter_guess[k] parameter_fit[k]=p if verbose: print_guesses(parameter_guess, parameter_lowerlimit, parameter_upperlimit, parameter_fit) return parameter_fit, redchi2gauss2, mpperr
def get_bestfit_sfms_groupcat(Mrcut=18, fid_mass=10.5, clobber=False): ''' Calculate the linear bestfit parameters for the StarForming Main Sequence of the SDSS Group Catalog specified by Mrcut or SDSS+PRIMUS envcount catalog from qf_env project. Fitting is done using MPFit ---------------------------------------------------------------- Parameters ---------------------------------------------------------------- Mrcut : absolute magntiude cut that specifies the group catalog clobber : Rewrite if True ---------------------------------------------------------------- Notes ---------------------------------------------------------------- * Bestfit values are accessed from file, unless it doesn't exist or clobber == True ''' bestfit_file = ''.join([ 'dat/central_quenching/sf_ms/' 'sf_ms_fit_starforming_groupcat_', str(Mrcut), '.hdf5' ]) if Mrcut == 18: z_med = 0.03 elif Mrcut == 19: z_med = 0.05 elif Mrcut == 20: z_med = 0.08 if not os.path.isfile(bestfit_file) or clobber: print 'Writing ' print bestfit_file avg_sfrs, var_sfrs, ngal = get_sfr_mstar_z_groupcat( np.arange(9.0, 11.5, 0.25), Mrcut=Mrcut ) enough_gal = np.where(ngal > 100) masses = np.arange(9.0, 11.5, 0.25 )[enough_gal] avg_sfrs = np.array(avg_sfrs)[enough_gal] var_sfrs = np.array(var_sfrs)[enough_gal] p0 = [0.5607, 0.0775917] # guess fa = { 'x': np.array(masses) - fid_mass, 'y': np.array(avg_sfrs), 'err': np.array(var_sfrs) } bestfit = mpfit.mpfit( mpfit_line, p0, functkw = fa ) # save data to h5py file f = h5py.File(bestfit_file, 'w') grp = f.create_group('slope_yint') grp.attrs['fid_mass'] = fid_mass grp.create_dataset('zmid', data=[z_med]) grp.create_dataset('slope', data=[0.65]) grp.create_dataset('yint', data=[0.0]) #grp.create_dataset('slope', data=[bestfit.params[0]]) #grp.create_dataset('yint', data=[bestfit.params[1]]) f.close() #return [0.1, bestfit.params[0], bestfit.params[1]] return [z_med, 0.65, 0.0] else: f = h5py.File(bestfit_file, 'r') zmids = f['slope_yint/zmid'][:] slopes = f['slope_yint/slope'][:] yints = f['slope_yint/yint'][:] f.close() return [zmids, slopes, yints]
def get_bestfit_sfms_envcount(fid_mass = 10.5, clobber = False): ''' Calculate linear bestfit parameters for SF-MS fits for redshift bins with delta z = 0.2. The slope and y-int are fit for z ~ 0.1. While only y-int is fit for z > 0.2 ''' bestfit_file = ''.join([ 'dat/central_quenching/sf_ms/' 'sf_ms_fit_starforming_envcount.hdf5' ]) if not os.path.isfile(bestfit_file) or clobber: print 'Writing ' print bestfit_file # first calculate average SFR, sigma_sfr as a function of mass # for SDSS redshift bin of the envcount catalog. Afterwards # fit slope and yint to the SF-MS. Notes that mass range is higher # than later in the function. avg_sfrs, sig_sfrs, ngal = get_sfr_mstar_z_envcount( np.arange(9.0, 11.5, 0.25), [0.1 for i in xrange(len(np.arange(9.0, 11.5, 0.25)))] ) enough_gal = np.where(np.array(avg_sfrs) != -10.) masses = np.arange(9.0, 11.5, 0.25 )[enough_gal] avg_sfrs = np.array(avg_sfrs)[enough_gal] sig_sfrs = np.array(sig_sfrs)[enough_gal] p0 = [0.0775917] # guess fa = { 'x': np.array(masses) - fid_mass, 'y': np.array(avg_sfrs), 'err': np.array(sig_sfrs) } bestfit = mpfit.mpfit( mpfit_line_fixedslope, p0, functkw = fa ) sdsszbin_slope = 0.65 #bestfit.params[0] sdsszbin_yint = 0.2 #bestfit.params[0] # use bestfit slope of low z SDSS bin to fit fixed slope # lines to the rest of the redshift bins zmids, slopes, yints = [], [], [] zbins = [0.3, 0.5, 0.7, 0.9] for zbin in zbins: avg_sfrs, sig_sfrs, ngal = get_sfr_mstar_z_envcount( np.arange(9.0, 11.5, 0.25), [zbin for i in xrange(len(np.arange(9.0, 11.5, 0.25)))] ) enough_gal = np.where(np.array(avg_sfrs) > -10.) masses = np.arange(9.0, 11.5, 0.25 )[enough_gal] avg_sfrs = np.array(avg_sfrs)[enough_gal] sig_sfrs = np.array(sig_sfrs)[enough_gal] p0 = [0.5] # bad guess fa = { 'slope': sdsszbin_slope, 'x': np.array(masses) - fid_mass, 'y': np.array(avg_sfrs), 'err': np.array(sig_sfrs) } bestfit = mpfit.mpfit( mpfit_line_fixedslope, p0, functkw = fa, quiet=1 ) zmids.append(zbin) slopes.append(sdsszbin_slope) yints.append(bestfit.params[0]) zmids.insert(0, 0.1) slopes.insert(0, sdsszbin_slope) yints.insert(0, sdsszbin_yint) f = h5py.File(bestfit_file, 'w') grp = f.create_group('slope_yint') grp.attrs['fid_mass'] = fid_mass grp.create_dataset('zmid', data=zmids) grp.create_dataset('slope', data=slopes) grp.create_dataset('yint', data=yints) f.close() return [zmids, slopes, yints] else: f = h5py.File(bestfit_file, 'r') zmids = f['slope_yint/zmid'][:] slopes = f['slope_yint/slope'][:] yints = f['slope_yint/yint'][:] f.close() return [zmids, slopes, yints]
def fitGauss(xdata, ydata, yerr, flatLine=False): nBins = 100 amplitude = 0.5 * np.max(ydata) x_offset = xdata[np.argmax(ydata)] sigma = (np.max(xdata) - np.min(xdata)) / 10.0 y_offset = 3.0 fixed = [False] * 4 if flatLine == True: amplitude = 0 fixed[0:3] = [True] * 3 params = [sigma, x_offset, amplitude, y_offset] # First guess at fit params errs = yerr errs[np.where(errs == 0.0)] = 1.0 quiet = True parinfo = [ { "n": 0, "value": params[0], "limits": [0.0001, 0.1], "limited": [True, True], "fixed": fixed[0], "parname": "Sigma", "error": 0, }, { "n": 1, "value": params[1], "limits": [x_offset - sigma * 3, x_offset + sigma * 3], "limited": [True, True], "fixed": fixed[1], "parname": "x offset", "error": 0, }, { "n": 2, "value": params[2], "limits": [0.2 * amplitude, 3.0 * amplitude], "limited": [True, True], "fixed": fixed[2], "parname": "Amplitude", "error": 0, }, {"n": 3, "value": params[3], "limited": [False, False], "fixed": fixed[3], "parname": "y_offset", "error": 0}, ] fa = {"x": xdata, "y": ydata, "err": yerr} m = mpfit.mpfit(gaussian, functkw=fa, parinfo=parinfo, maxiter=1000, quiet=quiet) if m.status <= 0: print m.status, m.errmsg mpp = m.params # The fit params mpperr = m.perror for k, p in enumerate(mpp): parinfo[k]["value"] = p parinfo[k]["error"] = mpperr[k] # print parinfo[k]['parname'],p," +/- ",mpperr[j] if k == 0: sigma = p if k == 1: x_offset = p if k == 2: amplitude = p if k == 3: y_offset = p fineXdata = np.linspace(np.min(xdata), np.max(xdata), 100.0) gaussfit = y_offset + amplitude * np.exp(-((xdata - x_offset) ** 2) / (2.0 * (sigma ** 2))) fineGaussFit = y_offset + amplitude * np.exp(-((fineXdata - x_offset) ** 2) / (2.0 * (sigma ** 2))) resolution = np.abs(x_offset / (2.355 * sigma)) return { "gaussfit": gaussfit, "resolution": resolution, "sigma": sigma, "x_offset": x_offset, "amplitude": amplitude, "y_offset": y_offset, "fineXdata": fineXdata, "fineGaussFit": fineGaussFit, "parinfo": parinfo, }