NLP4RE - Paper Session 01 - Hybrid session

Open Challenges in NLP for NFRs: A focus on Semantics, Generatlization, and Interpretability.
Rrezarta Krasniqi

Key research question: What are the key challenges to leveraging existing NLP models within the NFR domain, with respect to semantic soundess, ontology generalizability and output interpretability.

The system shall ensure high security for financial transactions.

Ambiguity: encryption, access control... etc. What exacltly does high security means.

*This work connects really well with OBRE. We should contact her to collaborate. Many well-illustrated problems:

The website shall have low latency response times during high traffic events, ensuring smooth user experience. Current NLP models fail to capture the context of slow since it is only inferred.

She proposes a roadmap concerned with three diferent aspects:

Each of the aspects is presented and illustrated: Ontology generalizability:

Interpretability:

------------- Main motivations:

They just applied GPT because the company had a license. There was no comparison with other LLMs. There was a question about this, claiming that perhaps ChatGPT costs are not needed. Applied approach: empirical studies (4 studies)

In general:

Study 1

Precision is very high, but recall is moderate, meaning tha ChatGPT missed quite some instances.

Study 2:

Impression of the engineers (there is no ground truth for this study, so the assessment is quite subjective

*All the content of these studies are contained in a GitHub link presented in the paper.