Forecasting Future Action Sequences with Neural Memory Networks

Paper

Forecasting Future Action Sequences with Neural Memory Networks

Published Sep 1, 2019 · Harshala Gammulle, Simon Denman, S. Sridharan

34

Citations

5

Influential Citations

PDF

Abstract

We propose a novel neural memory network based framework for future action sequence forecasting. This is a challenging task where we have to consider short-term, within sequence relationships as well as relationships in between sequences, to understand how sequences of actions evolve over time. To capture these relationships effectively, we introduce neural memory networks to our modelling scheme. We show the significance of using two input streams, the observed frames and the corresponding action labels, which provide different information cues for our prediction task. Furthermore, through the proposed method we effectively map the long-term relationships among individual input sequences through separate memory modules, which enables better fusion of the salient features. Our method outperforms the state-of-the-art approaches by a large margin on two publicly available datasets: Breakfast and 50 Salads.

Study Snapshot

Our neural memory network-based framework effectively forecasts future action sequences, outperforming state-of-the-art approaches on two public datasets.

PopulationOlder adults (50-71 years)

Sample size24

MethodsObservational

OutcomesBody Mass Index projections

ResultsSocial networks mitigate obesity in older groups.

Forecasting Future Action Sequences with Neural Memory Networks

References

Citations