Fine-tuning LLMs for answer set programming.
Saved in:
| Title: | Fine-tuning LLMs for answer set programming. |
|---|---|
| Authors: | Coppolillo, Erica1,2 (AUTHOR) erica.coppolillo@unical.it, Calimeri, Francesco1,3 (AUTHOR) francesco.calimeri@unical.it, Manco, Giuseppe2 (AUTHOR) giuseppe.manco@icar.cnr.it, Perri, Simona1 (AUTHOR) simona.perri@unical.it, Ricca, Francesco1 (AUTHOR) francesco.ricca@unical.it |
| Source: | Journal of Intelligent Information Systems. Apr2026, Vol. 64 Issue 2, p653-685. 33p. |
| Subjects: | Logic programming, Code generators, Language models, Model validation, Machine learning, Constraint programming |
| Abstract: | Large Language Models (LLMs) have demonstrated impressive capabilities across a wide range of natural language processing tasks, including code generation. While substantial progress has been made in adapting LLMs to generate code for various imperative programming languages, their effectiveness in handling declarative paradigms, such as Answer Set Programming (ASP), remains largely underexplored. This paper takes a step toward bridging that gap by investigating the potential of LLMs for ASP code generation. We begin with a systematic evaluation of several foundational LLMs, moving towards state-of-the-art models. We show that, despite their extensive training, large parameter counts, and significant computational backing, older models exhibit poor performance in generating syntactically and semantically correct ASP programs, while most recent ones mainly achieve impressive results. However, to overcome the need for huge computational power, we introduce LLASP, a fine-tuned, lightweight model specifically trained to encode ASP programs. In this regard, we extensively explore the effectiveness of fine-tuning by curating several dedicated datasets suitable for ASP encoding with increasing levels of complexity. First, we show that LLASP is effective in encoding template-based core problems in ASP; second, that the training strategy can be pushed forward to disregard the need for templating and make the generation prompt-invariant; and lastly, we show that even complex problems can be effectively encoded, beyond core tasks. Experimental results also show that LLASP significantly outperforms both its non-fine-tuned counterparts and most general-purpose LLMs, particularly in terms of semantic correctness, achieving a good trade-off between accuracy and resource-efficiency. Experimental code is publicly available at: https://github.com/EricaCoppolillo/LLASP. [ABSTRACT FROM AUTHOR] |
| Copyright of Journal of Intelligent Information Systems is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
| FullText | Text: Availability: 0 |
|---|---|
| Header | DbId: egs DbLabel: Engineering Source An: 193495188 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Fine-tuning LLMs for answer set programming. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Coppolillo%2C+Erica%22">Coppolillo, Erica</searchLink><relatesTo>1,2</relatesTo> (AUTHOR)<i> erica.coppolillo@unical.it</i><br /><searchLink fieldCode="AR" term="%22Calimeri%2C+Francesco%22">Calimeri, Francesco</searchLink><relatesTo>1,3</relatesTo> (AUTHOR)<i> francesco.calimeri@unical.it</i><br /><searchLink fieldCode="AR" term="%22Manco%2C+Giuseppe%22">Manco, Giuseppe</searchLink><relatesTo>2</relatesTo> (AUTHOR)<i> giuseppe.manco@icar.cnr.it</i><br /><searchLink fieldCode="AR" term="%22Perri%2C+Simona%22">Perri, Simona</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> simona.perri@unical.it</i><br /><searchLink fieldCode="AR" term="%22Ricca%2C+Francesco%22">Ricca, Francesco</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> francesco.ricca@unical.it</i> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Journal+of+Intelligent+Information+Systems%22">Journal of Intelligent Information Systems</searchLink>. Apr2026, Vol. 64 Issue 2, p653-685. 33p. – Name: Subject Label: Subjects Group: Su Data: <searchLink fieldCode="DE" term="%22Logic+programming%22">Logic programming</searchLink><br /><searchLink fieldCode="DE" term="%22Code+generators%22">Code generators</searchLink><br /><searchLink fieldCode="DE" term="%22Language+models%22">Language models</searchLink><br /><searchLink fieldCode="DE" term="%22Model+validation%22">Model validation</searchLink><br /><searchLink fieldCode="DE" term="%22Machine+learning%22">Machine learning</searchLink><br /><searchLink fieldCode="DE" term="%22Constraint+programming%22">Constraint programming</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: Large Language Models (LLMs) have demonstrated impressive capabilities across a wide range of natural language processing tasks, including code generation. While substantial progress has been made in adapting LLMs to generate code for various imperative programming languages, their effectiveness in handling declarative paradigms, such as Answer Set Programming (ASP), remains largely underexplored. This paper takes a step toward bridging that gap by investigating the potential of LLMs for ASP code generation. We begin with a systematic evaluation of several foundational LLMs, moving towards state-of-the-art models. We show that, despite their extensive training, large parameter counts, and significant computational backing, older models exhibit poor performance in generating syntactically and semantically correct ASP programs, while most recent ones mainly achieve impressive results. However, to overcome the need for huge computational power, we introduce LLASP, a fine-tuned, lightweight model specifically trained to encode ASP programs. In this regard, we extensively explore the effectiveness of fine-tuning by curating several dedicated datasets suitable for ASP encoding with increasing levels of complexity. First, we show that LLASP is effective in encoding template-based core problems in ASP; second, that the training strategy can be pushed forward to disregard the need for templating and make the generation prompt-invariant; and lastly, we show that even complex problems can be effectively encoded, beyond core tasks. Experimental results also show that LLASP significantly outperforms both its non-fine-tuned counterparts and most general-purpose LLMs, particularly in terms of semantic correctness, achieving a good trade-off between accuracy and resource-efficiency. Experimental code is publicly available at: https://github.com/EricaCoppolillo/LLASP. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Journal of Intelligent Information Systems is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=193495188 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1007/s10844-025-01017-4 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 33 StartPage: 653 Subjects: – SubjectFull: Logic programming Type: general – SubjectFull: Code generators Type: general – SubjectFull: Language models Type: general – SubjectFull: Model validation Type: general – SubjectFull: Machine learning Type: general – SubjectFull: Constraint programming Type: general Titles: – TitleFull: Fine-tuning LLMs for answer set programming. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Coppolillo, Erica – PersonEntity: Name: NameFull: Calimeri, Francesco – PersonEntity: Name: NameFull: Manco, Giuseppe – PersonEntity: Name: NameFull: Perri, Simona – PersonEntity: Name: NameFull: Ricca, Francesco IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 04 Text: Apr2026 Type: published Y: 2026 Identifiers: – Type: issn-print Value: 09259902 Numbering: – Type: volume Value: 64 – Type: issue Value: 2 Titles: – TitleFull: Journal of Intelligent Information Systems Type: main |
| ResultId | 1 |
