Official source code repository for "Deep Learning with Swift for TensorFlow" book.
🛍 @ Amazon (Paperback / Kindle) / Books / Apress / SpringerLink
Discover more insight about deep learning algorithms with Swift for TensorFlow. The Swift language was designed by Apple for optimized performance and development whereas TensorFlow library was designed by Google for advanced machine learning research. Swift for TensorFlow is a combination of both with support for modern hardware accelerators and more. This book covers the deep learning concepts from fundamentals to advanced research. It also introduces the Swift language for beginners in programming. This book is well suited for newcomers and experts in programming and deep learning alike. After reading this book you should be able to program various state-of-the-art deep learning algorithms yourself.
The book covers foundational concepts of machine learning. It also introduces the mathematics required to understand deep learning. Swift language is introduced such that it allows beginners and researchers to understand programming and easily transit to Swift for TensorFlow, respectively. You will understand the nuts and bolts of building and training neural networks, and build advanced algorithms.
- Understand deep learning concepts
- Program various deep learning algorithms
- Run the algorithms in cloud
- Newcomers to programming and/or deep learning, and experienced developers.
- Experienced deep learning practitioners and researchers who desire to work in user space instead of library space with a same programming language without compromising the speed.
1.1 Machine Learning
1.1.1 Experience
1.1.2 Task
1.1.3 Performance Measure
1.2 Machine Learning Paradigms
1.2.1 Supervised Learning
1.2.2 Unsupervised Learning
1.2.3 Semi-supervised Learning
1.2.4 Reinforcement Learning
1.3 Maximum Likelihood Estimation
1.4 Elements of a Machine Learning Algorithm
1.4.1 Data
1.4.2 Models
1.4.3 Loss Function
1.4.4 Optimizer
1.4.5 Regularizer
1.5 Bias and Variance Trade-Off
1.6 Why Deep Learning?
1.6.1 Curse of Dimensionality
1.6.2 Invalid Smoothness Assumption
1.6.3 Deep Learning Advantages
1.7 Summary
2.1 Linear Algebra
2.1.1 Matrices and Vectors
2.1.2 Unary Matrix Operations
2.1.3 Binary Matrix Operations
2.1.4 Norms
2.2 Probability Theory
2.2.1 Joint Probability
2.2.2 Conditional Probability
2.2.3 Elementary Rules
2.2.4 Chain Rule
2.2.5 Bayes Rule
2.3 Differential Calculus
2.3.1 Function
2.3.2 Differentiation of Univariate Function
2.3.3 Differentiation of Multivariate Function
2.3.4 Differentiation of Vector Function
2.3.5 Differentiation of Matrix Function
2.4 Summary
3.1 Swift is Everywhere
3.2 Swift for TensorFlow
3.3 Algorithmic Differentiation
3.3.1 Programming Approaches
3.3.2 Accumulation Modes
3.3.3 Implementation Approaches
3.4 Swift Language
3.4.1 Values
3.4.2 Collections
3.4.3 Control Flow
3.4.4 Closures and Functions
3.4.5 Custom Types
3.4.6 Modern Features
3.4.7 Error Handling
3.4.8 Advanced Operators
3.4.9 Differentiation
3.5 Python Interoperability
3.6 Summary
4.1 Tensor
4.2 Dataset Loading
4.2.1 Epochs and Batches
4.3 Defining Model
4.3.1 Neural Network Protocols
4.3.2 Sequence of Layers
4.4 Training and Testing
4.4.1 Checkpointing
4.4.2 Model Optimization
4.4.3 TrainingLoop
4.5 From Scratch for Research
4.5.1 Layer
4.5.2 Activation Function
4.5.3 Loss Function
4.5.4 Optimizer
4.6 Summary
5.1 Gradient-Based Optimization
5.1.1 Maxima, Minima, and Saddle Points
5.1.2 Input Optimization
5.1.3 Parameters Optimization
5.2 Linear Models
5.2.1 Regression
5.2.2 Classification
5.3 Deep Neural Network
5.3.1 Dense Neural Network
5.4 Activation Functions
5.4.1 Sigmoid
5.4.2 Softmax
5.4.3 ReLU
5.4.4 ELU
5.4.5 Leaky ReLU
5.4.6 SELU
5.5 Loss Functions
5.5.1 Sum of Squares
5.5.2 Sigmoid Cross-Entropy
5.5.3 Softmax Cross-Entropy
5.6 Optimization
5.6.1 Gradient Descent
5.6.2 Momentum
5.7 Regularization
5.7.1 Dataset
5.7.2 Architecture
5.7.3 Loss Function
5.7.4 Optimization
5.8 Summary
6.1 Convolutional Neural Network
6.1.1 Convolution Layer
6.1.2 Dimensions Calculation
6.1.3 Pooling Layer
6.1.4 Upsampling
6.2 Prominent Features
6.2.1 Local Connectivity
6.2.2 Parameter Sharing
6.2.3 Translation Equivariance
6.3 Shortcut Connection
6.4 Image Recognition
6.5 Conclusion
- First, you should install the latest Swift for TensorFlow toolchain.
- But to run only differentiation-specific code (for instance, the source code of Differentiable Programming chapter) not requiring deep learning features, simply install the latest Swift toolchain snapshot from Swift.org under Trunk Development (main) section. (The future differentiation feature updates will be posted on this website and will go through the standard Swift Evolution process.)
- Then you must select the newly installed toolchain in Xcode from
Preferences (Command + ,) > Components > Toolchains > (Swift for TensorFlow or Swift Development Snapshot).
This Swift package offers various executable targets listed in Package.swift.
List of all executable targets:
AdvancedOperatorsAlgorithmicDifferentiationArraysClassesClosuresConditionalStatementsControlTransferDictionariesDifferentiationEarlyExitEnumerationsErrorHandlingExtensionsGenericsGlobalFunctionsLoopsNestedFunctionsProtocolsPythonInteroperabilitySetsStructuresValues
EpochAndBatchesFromScratchForResearchModelDefinitionTensorExplanationTrainingAndTestingTrainingLoopExample
InputOptimizationLinearRegressionParametersOptimizationPolynomialRegression
ImageRecognition
Select the executable target (for instance, AlgorithmicDifferentiation) from the scheme drop-down menu and then click the Run button. And result should be presented in the console.
To execute any of them first enter into this package's root directory.
cd deep-learning-with-swift-for-tensorflow-book
Then run the following command in terminal.
swift run AlgorithmicDifferentiation
The following output will be displayed.
expression value: 30.0
expression derivative: 28.0
Just replace AlgorithmicDifferentiation with any executable target you wish to run.
Note: This Swift package was tested on Xcode 12.5 running on macOS version 11.3. It should also work right out of the box on Linux distributions.