Flow is a mental state experienced during holistic involvement in a certain task, and it is a factor that promotes motivation, development, and performance. A reliable and objective estimation of the flow is essential for moving away from the traditional self-reporting subjective questionnaires, and for developing closed-loop human-computer interfaces. In this study, we recorded EEG and pupil dilation in a cohort of participants solving arithmetic problems. In particular, the EEG activity was acquired with a prototype of a commercial headset from Logitech with nine dry electrodes incorporated in a pair of over-ear headphones. The difficulty of the tasks was adapted to induce mental Boredom, Flow and Overload, corresponding to too easy, optimal and too challenging tasks, respectively. Results indicated statistically significant differences between all pairs of conditions for the pupil dilation, as well as for the EEG activity for the electrodes in the ear-pads. Furthermore, we built a predictive model that estimated the mental state of the user from their EEG data with 65% accuracy.