#FP64 GPU NEURAL NETWORKS PC#
The current AMX design has been in such flux, so many new features added each year, and it's clear that the initial instruction set has not grown well into the new functionality! So I expect them at some point, when the pace of change slows down, to fix/redesign the instruction set and perhaps, at that point, expose it to the compiler.Just a decade ago, if you wanted access to a GPU to accelerate your data processing or scientific simulation code, you’d either have to get hold of a PC gamer or contact your friendly neighborhood supercomputing center.
So a great facility for math/sci/eng type use cases - especially when it can become a direct compiler target. (Which is not to say it's worthless! The latest patents to my eyes suggest it is being turned into a kinda super AVX512, still the matrix functionality but also a lot of the FP compute abilities of AVX512 without the downsides. During inference, you might go down to even int8, but that requires advanced modeling tricks.įor what is worth, a lot of ML inference is still done on devices with no dedicated accelerator so work in AVX512 optimization for ML (Intel MLK framework being a notable example) is an active area of interest for companies. It terms of precision, training is done either with fp32 precision, or in mixed precision mode (fp32 + fp16) double floats are unnecessary. In general, different parts of a platform (CPU, GPU, various accelerators) are transparently optimizing by the framework you are using. From what I've briefly read, since it leverage Metal APIs, I would assume that it also keep advantage of the AMX if not, I'm sure that's in the pipeline. They can also fuse some operators to get better performance. I haven't looked specifically into how the MPS backend used for M1 devices works on CUDA, PyTorch and other frameworks blend different cores (Tensor cores, generic CUDA cores), depending on the operations. The current AMX design has been in such flux, so many new features added each year, and it's clear that the initial instruction set has not grown well into the new functionality! So I expect them at some point, when the pace of change slows down, to fix/redesign the instruction set and perhaps, at that point, expose it to the compiler.) or perhaps AMX is no longer really part of ML anymore? neural networks have many layers, and some of them are best handled on a GPU, some on AMX, some even on a CPU? (I've suggested doing this, in a different context, for large matrix manipulation like eigenvalues or solving elliptical PDEs) we run the first phases of training on the GPU to get close to convergence, then the final phases at higher accuracy on AMX. Once the design is stabilized, you know how much and where you can move it to 16b or even 8b and GPU is better. AMX is useful in the exploratory phases of designing a network, where you don't know what you're doing or how it will work, and 32b, even 64b accuracy, is helpful and easy.
So what's the role of AMX (introduced for the purposes of ML, and with the first few patent updates very much ML-focussed) in all this?Īnswers I could imagine (but I don't know): It *seems* that, even on the M1, all the major ML frameworks are bifurcating into infer on the Apple NPU, and train on the Apple GPU. Click to expand.I assume you are a practitioner.
0 kommentar(er)
