In Math what are Tensors, as in TensorFlow?
In: 0
In math, we like to make things easy. Probably the easiest thing we can work with that still actually do something are lines. Even easier still are lines that go through the origin. These have the equation y=mx. Trying to turn things into lines is basically the idea behind derivatives in calculus, and differential equations are useful because of this. So lines are pretty simple, but also really useful.
But the line y=mx is only in one dimension. One dimension in, one dimension out. If your input is 2D, 3D or bigger than 80Dimensions – not unheard of when doing Data Analysis – then you will need something that can work in these dimensions. This is what Linear Algebra is for. Matricies are how we do the equation y=mx, but in higher dimensions as Ax=b. You can input an N-dimensional piece of data and get out an M-dimensional piece of data. Consequently, Linear Algebra is probably the most important modern field of math out there. You can analyze signals, do image processing, facial recognition, and a whole lot of other very important things with computers using little more than Linear Algebra.
An important application of Linear Algebra – the Ax=b equation – is Linear Regression. What happens here is that you have a set of data whose inputs and outputs you think should be linear. Linear regression finds the best possible matrix A so that whenever x is an input for your data, then Ax will be as close – on average – as it can be to the output data corresponding to x. This is the basis for much of Machine Learning. All the neural networks and stuff are just multiple, linked instances of Linear Regression tied together. It’s a web of finding the right matrix that will fit your data in the way you think is the best. There’s a lot of complexity that you can control, which is why it works so well.
But matricies are limited because you can only input one thing at a time. One vector in, one vector out. Tensors expand this. Instead of Ax=b, you have T(x,y,z)=S where x,y,z are vectors of whatever dimension you want (and you can extend well beyond z if you like) and, in a way, S can be thought of as multiple vectors being output. This gives a lot of flexibility and control and can make machine learning much more fluid. This goes beyond basic linear algebra into what we call “Multilinear Algebra”.
Now, in the simple y=mx, m is a single number. For Ax=b, A is a matrix which is a 2D array of numbers and the size of this array tells us about the dimensions of the inputs and outputs. To T(x,y,z,…)=S, it is a multidimensional array whose dimensions tell you about the number and dimensions of possible inputs and outputs. Derivatives find the right “m” to fit a curve. Linear Regression finds the right A to fit a data. And multilinear regression finds the right T.
But, the word “tensor” is way overused. Different mathematicians, depending on the field, will use it differently. Physicists use it differently. Engineers and Computer Scientists use it differently. They’re all related but have wildly different connotations. This is specific to how the word is used in TensorFlow.
Latest Answers