Skip to main content

Questions tagged [basis-function]

A basis function is an element of a set of functions that span a function space.

Filter by
Sorted by
Tagged with
5 votes
3 answers
303 views

Understanding spline transformation and regression coefficients

I do not understand properly what a spline does even in a simple situation of a piecewise regression, and I need some help. Consider the following basic example: ...
denis's user avatar
  • 191
3 votes
1 answer
102 views

Implications of keeping a "low" basis dimension in GAMM

Some of the smooths in my generalized additive mixed model (GAMM) indicate in mgcv::k.check(m) they want to be more wiggly, but I don't think I have enough data to ...
Nate's user avatar
  • 1,529
2 votes
0 answers
93 views

Basis dimension (k) low but unable to increase it

I have a GAM of the form: Abundance ~ Treatment + s(KelpCover) + s(DayofYear, by=Treatment) Where Abundance is a count of ...
Jordan Woolfrey's user avatar
5 votes
0 answers
117 views

How to transform basis functions to satisfy a restriction

I am trying to find a way to methodically transform a set of basis functions to satisfy a restriction so that the coefficient of the first basis function at a specific value of $x$ can have a special ...
Frank Harrell's user avatar
0 votes
1 answer
69 views

Empirical basis functions

Preliminary Consider $n$ individuals each with observed data $ Z_i, i = 1, \ldots, n$. For each individual $i$, the longitudinal predictor $Z_i = \{Z_i(t_{i1}), \ldots, Z_i(t_{i,R_i})\}$ is measured ...
ADAM's user avatar
  • 731
0 votes
1 answer
66 views

How to construct cardinal spline bases

I would like to know how to algebraically construct cardinal spline bases, as I would like to make a prediction with a natural cubic spline model which uses them; however, the only source I have found ...
Florent H's user avatar
  • 153
0 votes
0 answers
14 views

Short linear basis expansion for support vector regression

In support vector regression (SVR), I know that many different linear basis expansions for the predictors can be used. I am interested in a basis expansion that is as algebraically short as possible ...
Florent H's user avatar
  • 153
2 votes
0 answers
54 views

Interpretation of basis functions in a logistic regression: can we test for univariate and multivariate/copula differences between the categories?

O'Brien (1988) has shown that a strong method for doing multivariate testing is to reverse the problem. That is, instead of seeing if the category impacts the measured values, see how the measured ...
Dave's user avatar
  • 62.7k
5 votes
1 answer
296 views

Spline basis explicitly including a linear term; basis functions generated by the default call to "s()" function of mgcv package

I'm curious about the basis functions generated by the call to "s()" function with default parameter values, but even more specifically I'm curious about a smoother for a single variable ...
UsDAnDreS's user avatar
  • 139
1 vote
0 answers
87 views

EDF and basis functions in GAMs with factor-smooth interactions

I am trying to use a GAM to model average daily water temperature against day and habitat type in an estuary. I have temperature logger data over 115 days across three habitat types. I am using a GAM ...
Phoebe's user avatar
  • 11
3 votes
1 answer
272 views

Second differences notion behind GAM penalties

I'm going back through Simon Wood's book on generalized additive models (GAMs) and came back across the definition of the penalty term employed, which is supposed to combat overfitting of smooths. The ...
Shawn Hemelstrand's user avatar
1 vote
0 answers
52 views

Kernelization vs pre-defined basis functions: which one is better and why?

I am learning about kernels and how linear models can use them to model nonlinear data. Consider, for example, linear regression for nonlinear function $y(\textbf{x})$. The idea is to project the ...
Botond's user avatar
  • 217
0 votes
1 answer
47 views

Covariance matrix of linear estimator of basis

I am reading Elements of Statistical Learning, specifically section 5.2.2, an example on the South African Heart Disease dataset. The idea is to model the logit of the conditional probability of the ...
Max's user avatar
  • 113
0 votes
0 answers
70 views

Why do we multiply the centered data with eigen vector instead of taking inverse of the eigen vector while performing PCA

The question may sound stupid but I really don't understand the logic behind this. Whenever we do a PCA, we take a covariance matrix on the centered data and do eigen decomposition. In order to ...
Jacob Simon Areickal's user avatar
1 vote
1 answer
71 views

Linear regression using two different classes of basis functions

Let's say we have 1D data $Y = \{ y_i \mid y_i \in \mathbb{R} \}$, and regressors $X = \{ x_i \mid x_i \in \mathbb{R} \}$, and we try doing basis regression. Suppose we find that we can perfectly ...
mobeets's user avatar
  • 221
1 vote
1 answer
304 views

Effect of basis functions on the dimension of a linear regression model

In Scikit-learn I can use polynomial features to create polynomial linear regression models. Scikit-learn transforms my original ...
Featherball's user avatar
4 votes
2 answers
116 views

Determining spline basis dimension using Wood's statistical test

In Simon Wood's book Generalized Additive Models (2nd ed.) on page 243, he describes the following procedure for checking that the basis dimension is too small: Fortunately informal checking that the ...
cgmil's user avatar
  • 1,343
3 votes
1 answer
1k views

What is the relationship between knots and cubic spline basis functions in GAMs?

I do not understand the relationship between knots and basis functions in Generalized Additive Models (hereafter GAM). In Chapter 4 of S. Wood's book "Generalized Additive Models - An ...
Marco Plebani's user avatar
6 votes
2 answers
2k views

How to correctly use I-Splines for monotone non-decreasing/ increasing regression?

I have the following data to which I want to fit a monotone non-decreasing spline. ...
Mihai's user avatar
  • 259
2 votes
1 answer
131 views

How to calculate basis functions by hand

I am learning about natural splines and basis functions and am struggling with it a lot. I understand the concepts of knots being the part where first and second derivatives are equal on either side. ...
Hariboharry's user avatar
1 vote
0 answers
28 views

Parameter estimation for basis function model in Elements of Statistical Learning (ESL)

In the book Elements of Statistical Learning, section 2.8.3 describes Basis Functions, citing an example of a radial basis function as $f_{\theta}(x) = \sum_{m=1}^M \beta_M \sigma(\alpha_m'x + b_m)$, ...
bob's user avatar
  • 725
1 vote
1 answer
783 views

How do splines work when being used on the right side of an equation?

I have seen 2 ways of using splines: Spline as the primary model: Here, we use a spline to model y as a function of a single covariate x. That is, it is used as a regression model. The example in the ...
Chaos's user avatar
  • 113
1 vote
2 answers
1k views

Confusion between basis functions and SVM feature mapping

I'm new to Machine Learning. I have just read an article about basis functions. Apparently, the basis functions create a non-linear regression line to capture a variety of differently complicated ...
Đặng Huy Hoàng's user avatar
1 vote
1 answer
138 views

Functional Linear Regression without Orthonormal Basis for Prediction

What do you do in functional linear regression when for whatever reason you don't want to use an orthonormal basis expansion? In functional linear regression for scalar on function regression, one may ...
mlstudent's user avatar
  • 492
2 votes
1 answer
311 views

Save computation time with mgcv?

Suppose we have a generalized additive model that is formulated as gam(y~s(x,bs='tp',k=10),...) I need to repeat the computation many times (e.g., 1000) for the ...
bluepole's user avatar
  • 2,699
1 vote
0 answers
53 views

Bivariate basis functions with span invariant to rotation about $z$-axis

Consider the following functions defined over $x,y\in\mathbb{R}$: $f_0(x,y)=1$ $f_1(x,y)=x$ $f_2(x,y)=y$ These functions form a basis with three-dimensional span (the set of all non-vertical planes) ...
Museful's user avatar
  • 375
-1 votes
1 answer
2k views

How To Solve A Sequence Question [closed]

I'm taking a course on R on edx, but I have been stocked here for days. Please Which integer values are between the maximum and minimum heights? For example, if the minimum height is 10.2 and the ...
Sodiq's user avatar
  • 1
3 votes
1 answer
621 views

What is the expanded representation, $\phi(X)$, required to obtain the RBF kernel?

For the two-dimensional case, where $\boldsymbol X=[x_1, x_2]$ and its corresponding expanded represetation $\boldsymbol\phi(X)= [1, \sqrt2 x_1, \sqrt2 x_2, x_1^2, x_2^2, \sqrt2x_1x_2]$, we can ...
Nip's user avatar
  • 561
2 votes
1 answer
444 views

How should I understand smoothing in functional data analysis from a modelling perspective (specifically for temperature data)?

The specifics of this question are that I am looking at daily maximum and minimum temperature from the GHCND data set obtained from NOAA's API and I view the temperatures observed at days in a year as ...
cgmil's user avatar
  • 1,343
1 vote
0 answers
41 views

Difficulties with orthogonal density estimation

I am working on an implementation of an orthogonal density estimator, using the basis $$ \psi_0(t) = 1, \quad \psi_{2j}(t) = \sqrt{2}\text{cos}(2\pi j t), \quad \psi_{2j+1}(t) = \sqrt{2}\text{sin}(2\...
chris75's user avatar
  • 21
0 votes
0 answers
465 views

Demmler-Reinsch basis for smoothing splines

I have seen some papers about using the so-called Demmler-Reinsch basis for smoothing spline because it is a basis for natural spline space and also Sobolev space. For example, these papers: A ...
Ham82's user avatar
  • 125
3 votes
3 answers
6k views

Problems interpreting GAM output

I have been advised to run General Additive Models to be able to describe trends in my data, my data being animal harvest numbers by year. I have done so, but have a problem with interpreting the ...
Dag's user avatar
  • 155
2 votes
1 answer
489 views

Smoothing splines as basis expansion

I am following the discussion on chapter 5 of Elements of statistical learning which discusses basis expansion using splines. The data set I used is the Ozone data which can be found at :http://web....
Zhezhong Jiang's user avatar
4 votes
1 answer
3k views

Difference between s() and ti() terms in mgcv package when applied to one variable

I am using the mgcv package in R to fit logistic GAMs to survey data. In one of my models I use an interaction between two covariates. I am currently trying to fit ...
Joanne Ellison's user avatar
4 votes
1 answer
104 views

Are $h_i(x)=x^{-\alpha_i}$ okay basis functions for fitting?

I have some pairs of data ${(x_1,y_1),..., (x_n,y_n)}$ genereated by some process and would like to fit it with a function so that $y_i \approx \hat{f}(x_i)$. By plotting the $(X,Y)$ on a 2D plot, ...
Tom Bennett's user avatar
6 votes
2 answers
1k views

What is Dimension of basis in splines

From mgcv package, gam(for generalized additive model fit) function uses the parameter 'k' which is dimension of basis. Can anyone explain to me what does dimension of basis function means in spline ...
Naveen Gabriel's user avatar
4 votes
1 answer
358 views

Please correct my assumption on how regression trees work

I'm trying to understand how regression trees work, I've been experimenting with catboost and xgboost in python, and I'm getting results which I don't expect, can someone please clarify (and apologies ...
David Waterworth's user avatar
3 votes
0 answers
175 views

Adaptive knot selection for B-spline fitting

When fitting a B-spline for regression purposes I've seen a lot of cases where knots are fixed uniformly ,but in some situations this could lead to poor estimations because the behaviour of the curve ...
Chaos's user avatar
  • 431
2 votes
1 answer
4k views

Number of basis functions in natural cubic spline

According to ESL, natural cubic basic spline with $K$ knots is represented by $K$ basis function. However, the ns() function in R with ...
momomi's user avatar
  • 125
3 votes
0 answers
806 views

How to calculate the output of ns() function in R by hand

ns() function in R can generate the natural cubic spline basis matrix. I checked the reference but still do not know how to calculate such matrix by hand. For ...
cow3918's user avatar
  • 31
2 votes
1 answer
599 views

Kernel function from polynomial basis functions

In chapter 3 & 6 of Bishop's Pattern Recognition and Machine Learning, he showed that the equivalent kernel based on eqn (3.62) $$ k(x,x') = \beta \phi(x)^T (\alpha I + \beta \Phi^T \Phi )^{-1}\...
David KWH's user avatar
0 votes
1 answer
27 views

Generate funciton with specified bases

I am trying to simulate functional data using specified bases and a set of random coefficients. For example, I can generate a function on $[0,1]$ with Fourier basis like $$f(x) = \sum_{i=0}^k a_i sin(...
PatrickXie17's user avatar
0 votes
0 answers
365 views

How to measure the effect of each covariate in GLM

I have a GLM model with two variables (Z1 and Z2). I have also used some basis functions to estimate these variables. In other words, Z1 is substituted by 10 basis functions and Z2 with 12 basis ...
Mina's user avatar
  • 169
1 vote
3 answers
2k views

Why does it matter that bases are linearly independent?

I am taking an advanced linear algebra course and am once again confused by a lot of the concepts. I understand that the definition of a basis is a set of vectors that spans the vector space and is ...
jmoore00's user avatar
  • 369
7 votes
1 answer
3k views

Is spline basis orthogonal?

When we talk about the basis, we have the concept like orthogonal, unit length etc. for vectors. I think the same concept also exist in Fourier basis and Polynomial basis. But how about spline (Say ...
Haitao Du's user avatar
  • 36.9k
2 votes
1 answer
2k views

Gaussian basis function in Bayesian Linear Regression

I am learning about Bayesian Linear Regression from the book "Pattern Recognition And Machine Learning" (Bishop, Christopher M.). I want to recreate graphs from illustration 3.8 and this require to ...
aklime's user avatar
  • 21
5 votes
1 answer
1k views

In using a basis model to represent a cubic spline, what is the meaning of the truncated power basis function and its math in lay terms?

I am going over “Introduction to Statistical Learning” (James, et al). While I understand the concept of a spline (piecewise polynomial with continuous 1st and 2nd derivative at each knot), I’ve been ...
hundred_dolla_tea's user avatar
4 votes
0 answers
373 views

Regression Spline basis functions

I'm reading this book - Generalized Additive Models by Simon Wood. On page 124, he defines basis for regression spline as (language is R) ...
user2696565's user avatar
  • 1,389
7 votes
1 answer
2k views

What are periodic version of splines?

In this What's wrong to fit periodic data with polynomials? post, I tried to use Fourier basis expansion and Polynomial basis expansion to fit a toy periodic data (daily temperature data set). I ...
Haitao Du's user avatar
  • 36.9k
3 votes
1 answer
48 views

bad fit - nomenclature for breeds

Question: What is it called when one uses a basis, like the pure line instead of the sigmoid/logistic, in a manner that grossly departs from the "physics" of the problem? There should be a word for ...
EngrStudent's user avatar
  • 9,375