2
$\begingroup$

I am learning Machine Learning (Linear Regression) from Prof. Andrew Ng's lecture. While listening when to use normal equation vs gradient descent, he says when our features number is very high (like $10^6$) then to use gradient descent.

Everything is clear to me, but I wonder that can someone give me real life examples where we use such such huge number of features?

a slide from ML coursera lecture

$\endgroup$
  • $\begingroup$ Where does the notion that there's a need to compute $(X^TX)^{-1}$ come from? That's not a good idea at all. $\endgroup$ – Glen_b -Reinstate Monica Apr 20 '14 at 22:10
  • $\begingroup$ Document classification, bioinformatics (microarray data, genomics, PPI networks, ...), web data mining (Twitter, Facebook, Google, ...), ... $\endgroup$ – Marc Claesen Apr 20 '14 at 22:16
4
$\begingroup$

Natural language processing come to mind. For instance, you might predict the amount of money someone spends on your website by their review. The review is text, encoded by an n-gram model. The ith, jth element of your training matrix is the ith customer and counts of jth n-gram. An n-gram is a string of contiguous words like, "the product was excellent."

Typically n-gram data is encoded in a sparse matrix. There are several data-structures suitable. One of the easiest to explain is a coordinate list. A coordinate list is a list of tuples [(i, j, value)]. Since the matrix is mostly zero, this is much more efficient than allocating a dense array.

$\endgroup$
3
$\begingroup$

Another example - image recognition. Imagine that you have just a 512 x 512 gray-scale image - it means that without additional pre-processing you already have $2^{18}$ features - with each pixel being a feature.

It's not necessarily a good example for Linear Regression, but Gradient Descent is used in many ML algorithms.

$\endgroup$
0
$\begingroup$

Genetic analysis is another common example. Most mammals have genomes with a number of protein-coding genes in the order of $10^4$, but there's lots of other coding regions (for different kinds of RNA that are not used for protein synthesis directly), too. Additionally, some experimental/methodological procedures bases on sequencing slice the genome into millions ($10^6$) of bins and measure certain molecular features for these bins (like DNA methylation or CpG-islands).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.