In my current role as a Natural Language Processing (NLP) Engineer at LinguaSoft, I've seen firsthand how important it is to maintain coherence in long-text generation. We were working on a project related to automating news article writing for a leading media company. Maintaining coherence in the generated articles was our primary challenge given the length and complexity of the topics.
We employed an attach-and-finetune process using the transformer-based GPT-3 model which is recognized for generating coherent and diverse texts. We also implemented a coherence score check which assisted us to fine-tune the model and discourage the generation of incoherent sentences.
Post-deployment, the media company was able to generate initial drafts of news articles automatically, with the outputs retaining the logical flow and coherence that's key for such pieces. The project was a significant success, with the client recording a 40% saving in the time spent on writing news articles.