In this talk, I will first briefly introduce our recent contributions to LLM training in Nancy, and then I will summarize a selection of the research works that have been carried out in our team to reduce the cost of LLM training, in particular with LLM compression, incremental training, PEFT and weakly-supervised training. All these approaches can be interpreted through a common lens: the loss landscape. I will adopt this point of view as a common thread to tie several of our cost reduction approaches together in my presentation.