I modified a Transformer encoder-decoder structure to work for time series and applied this model to the prediction of ocean currents. The embedding layer is replaced by a linear fully connected layer and a L2 loss is used instead of the softmax loss. I showed that the cross-attention weight map exhibits the harmonic patterns of the currents caused by the moon and the sun.

Ocean current data at 831 sites were downloaded from the historic dataset on NOAA website. The dataset was checked for data acquisition error and curated  so that it can be used by deep learning models. Transformer predictions were validated against a dataset of active stations and compared to predictions from a LSTM model.

The Transformer can run on an AUV's autopilot firmware and provide real-time in-situ prediction of ocean currents at any location using spatial data acquired by on-board sensors. These predictions can feed  its path planning and control systems to improve the safety and reliability of AUV navigation.