With the exponential growth in Artificial Intelligence (AI) in recent years, increasing applications have been observed across various fields, with oncology - the study of cancer - being no exception. The study of mathematical oncology involves the modelling and prediction of cancer growth and treatment response with the use of mathematical models. This appears to be a promising area for incorporating AI and machine learning (ML), but there exist many limitations to be analysed. This article focuses on the current applications of AI in oncology, alongside its limitations and future advances. The applications of AI in cancer research across various areas such as mathematical oncology as well as diagnosis, prognosis, treatment, and drug discovery in oncology are analysed. The ethical implications of using patient data for training AI models, as well as the possibly limited data available for model training, alongside means for preserving patient confidentiality are also discussed. The article concludes that there are numerous promising current and future applications of AI in oncology and mathematical oncology. However, the field as a whole suffers various limitations: some mitigated by the use of AI and some arising from it.