Convergence Analysis of Fractional Gradient Descent

The article explores the concept of fractional derivatives, which are mathematical tools used to model complex phenomena in various fields, including physics, engineering, and finance. Fractional derivatives are defined as generalizations of traditional derivative concepts, allowing for non-local properties and non-differentiable functions. However, these derivatives are not easily computed or integrated, leading to the development of different approximation methods and numerical schemes.
The author provides a comprehensive overview of various fractional derivative definitions, including the Caputo derivative and the Riemann-Liouville derivative. The article also discusses the challenges in computing and approximating fractional derivatives, such as the need for special functions and the existence of multiple representations.
To address these challenges, the author presents a set of numerical schemes and approximation methods, including the Feynman formula, the Laplace transform, and the Fourier transform. These methods provide efficient ways to compute fractional derivatives and integrate fractions of order α, where α is a real number.
The article also discusses the application of fractional derivatives in optimization algorithms, such as gradient descent and Adam optimization. The author shows how these algorithms can be modified to use fractional derivatives instead of traditional derivatives, leading to more efficient and robust optimization processes.
Throughout the article, the author uses simple analogies and explanations to demystify complex concepts. For instance, the author compares the behavior of fractional derivatives to a car driving on a non-linear road with many curves, where the traditional derivative would only capture the overall slope of the road.
In summary, the article provides a comprehensive overview of fractional derivatives, their definitions, and their applications in optimization algorithms. The author demystifies complex concepts by using everyday language and engaging analogies, making the article accessible to a wide range of readers, including those without advanced mathematical knowledge.