Publication
Refining english writing proficiency assessment and placement in developmental education using NLP tools and machine learning
| Summary: | This study investigates the enhancement of English writing proficiency assessment and placement for Developmental Education (DevEd) within U.S. colleges using Natural Language Processing (NLP) and Machine Learning (ML). Existing automated placement tools, such as ACCUPLACER, often lack transparency and struggle to identify nuanced linguistic features necessary for accurate skill-level classification. By integrating human-annotated linguistic features, this study aims to contribute to equitable and transparent placement systems that better address students’ academic needs, reducing misplacements and their associated costs. For this study, a 300-essay corpus was compiled and manually annotated with a refined set of 11 DevEdspecific (DES) features, alongside 328 linguistic features automatically extracted from CTAP and 106 via COH-METRIX. Supervised ML algorithms were used to compare ACCUPLACER-generated classifications with human ratings, assessing classification accuracy and identifying predictive features. This analysis revealed gaps in ACCUPLACER’s classification capabilities. Experimental results showed that models incorporating DES features improved classification accuracy, with Na¨ıve Bayes (NB) and Support Vector Machine (SVM) achieving scores up to 80%. The refined features presented and methodology offer actionable insights for faculty and institutions, potentially contributing to more effective DevEd course placements and targeted instructional interventions. |
|---|---|
| Main Authors: | Da Corte, Miguel |
| Other Authors: | Baptista, Jorge |
| Subject: | Developmental education (DevEd) Automatic writing assessment systems Natural language processing (NLP) Machine-learning models |
| Year: | 2025 |
| Country: | Portugal |
| Document type: | conference output |
| Access type: | open access |
| Associated institution: | Universidade do Algarve |
| Language: | English |
| Origin: | Sapientia - Universidade do Algarve |
| _version_ | 1877785681934155776 |
|---|---|
| author | Da Corte, Miguel |
| author2 | Baptista, Jorge |
| author2_role | author |
| author_facet | Da Corte, Miguel Baptista, Jorge |
| author_role | author |
| contributor_name_str_mv | Sapientia |
| country_str | PT |
| creators_json_txt | [{\"Person.name\":\"Da Corte, Miguel\",\"Person.identifier.orcid\":\"0000-0001-8782-8377\"},{\"Person.name\":\"Baptista, Jorge\",\"Person.identifier.cienciaID\":\"7010-5366-22C5\",\"Person.identifier.orcid\":\"0000-0003-4603-4364\",\"Person.identifier.researcherID\":\"H-7699-2013\",\"Person.identifier.scopusAuthorID\":\"14035269500\"}] |
| datacite.contributors.contributor.contributorName.fl_str_mv | Sapientia |
| datacite.creators.creator.creatorName.fl_str_mv | Da Corte, Miguel Baptista, Jorge |
| datacite.date.Accepted.fl_str_mv | 2025-01-01T00:00:00Z |
| datacite.date.available.fl_str_mv | 2026-06-02T16:14:56Z |
| datacite.date.embargoed.fl_str_mv | 2026-06-02T16:14:56Z |
| datacite.rights.fl_str_mv | http://purl.org/coar/access_right/c_abf2 |
| datacite.subjects.subject.fl_str_mv | Developmental education (DevEd) Automatic writing assessment systems Natural language processing (NLP) Machine-learning models |
| datacite.titles.title.fl_str_mv | Refining english writing proficiency assessment and placement in developmental education using NLP tools and machine learning |
| dc.contributor.none.fl_str_mv | Sapientia |
| dc.creator.none.fl_str_mv | Da Corte, Miguel Baptista, Jorge |
| dc.date.Accepted.fl_str_mv | 2025-01-01T00:00:00Z |
| dc.date.available.fl_str_mv | 2026-06-02T16:14:56Z |
| dc.date.embargoed.fl_str_mv | 2026-06-02T16:14:56Z |
| dc.format.none.fl_str_mv | application/pdf |
| dc.identifier.none.fl_str_mv | http://hdl.handle.net/10400.1/29074 |
| dc.language.none.fl_str_mv | eng |
| dc.publisher.none.fl_str_mv | SCITEPRESS - Science and Technology Publications |
| dc.rights.cclincense.fl_str_mv | http://creativecommons.org/licenses/by-nc-nd/4.0/ |
| dc.rights.none.fl_str_mv | http://purl.org/coar/access_right/c_abf2 |
| dc.subject.none.fl_str_mv | Developmental education (DevEd) Automatic writing assessment systems Natural language processing (NLP) Machine-learning models |
| dc.title.fl_str_mv | Refining english writing proficiency assessment and placement in developmental education using NLP tools and machine learning |
| dc.type.none.fl_str_mv | http://purl.org/coar/resource_type/c_c94f |
| description | This study investigates the enhancement of English writing proficiency assessment and placement for Developmental Education (DevEd) within U.S. colleges using Natural Language Processing (NLP) and Machine Learning (ML). Existing automated placement tools, such as ACCUPLACER, often lack transparency and struggle to identify nuanced linguistic features necessary for accurate skill-level classification. By integrating human-annotated linguistic features, this study aims to contribute to equitable and transparent placement systems that better address students’ academic needs, reducing misplacements and their associated costs. For this study, a 300-essay corpus was compiled and manually annotated with a refined set of 11 DevEdspecific (DES) features, alongside 328 linguistic features automatically extracted from CTAP and 106 via COH-METRIX. Supervised ML algorithms were used to compare ACCUPLACER-generated classifications with human ratings, assessing classification accuracy and identifying predictive features. This analysis revealed gaps in ACCUPLACER’s classification capabilities. Experimental results showed that models incorporating DES features improved classification accuracy, with Na¨ıve Bayes (NB) and Support Vector Machine (SVM) achieving scores up to 80%. The refined features presented and methodology offer actionable insights for faculty and institutions, potentially contributing to more effective DevEd course placements and targeted instructional interventions. |
| dirty | 0 |
| eu_rights_str_mv | openAccess |
| format | conferenceObject |
| fulltext.url.fl_str_mv | https://sapientia.ualg.pt/bitstreams/efc14f07-bdf3-4c2d-b76f-04a7f5fca9b6/download |
| funder_facet_str_mv | FCT{{{_:::_}}}Fundação para a Ciência e a Tecnologia |
| funding.funder.alternateName_str_mv | FCT |
| funding.funder.identifier_str_mv | http://doi.org/10.13039/501100001871 |
| funding.funder.name_str_mv | Fundação para a Ciência e a Tecnologia |
| funding.identifier_str_mv | UIDB/50021/2020 |
| funding.name_str_mv | 6817 - DCRRNI ID |
| funding_str_mv | UIDB/50021/2020 info:eu-repo/grantAgreement/FCT/6817 - DCRRNI ID/UIDB%2F50021%2F2020/PT |
| id | sapientia_daf79dfde64f739cffad4f2ea073e215 |
| identifier.url.fl_str_mv | http://hdl.handle.net/10400.1/29074 |
| inst_facet_str | urn:organizationAcronym:ualg{{{_:::_}}}Universidade do Algarve |
| instacron_str | ualg |
| institution | Universidade do Algarve |
| instname_str | Universidade do Algarve |
| language | eng |
| network_acronym_str | sapientia |
| network_name_str | Sapientia - Universidade do Algarve |
| oai_identifier_str | oai:sapientia.ualg.pt:10400.1/29074 |
| organization_str_mv | urn:organizationAcronym:ualg |
| person_str_mv | Da Corte, Miguel Da Corte, Miguel http://orcid.org/0000-0001-8782-8377 0000-0001-8782-8377 Baptista, Jorge Baptista, Jorge https://www.ciencia-id.pt/7010-5366-22C5 7010-5366-22C5 http://orcid.org/0000-0003-4603-4364 0000-0003-4603-4364 |
| publishDate | 2025 |
| publisher.none.fl_str_mv | SCITEPRESS - Science and Technology Publications |
| repo_facet_str | urn:repositoryAcronym:sapientia{{{_:::_}}}Sapientia - Universidade do Algarve |
| reponame_str | Sapientia - Universidade do Algarve |
| repository_id_str | urn:repositoryAcronym:sapientia |
| service_str_mv | urn:repositoryAcronym:sapientia |
| spelling | engSCITEPRESS - Science and Technology PublicationsengThis study investigates the enhancement of English writing proficiency assessment and placement for Developmental Education (DevEd) within U.S. colleges using Natural Language Processing (NLP) and Machine Learning (ML). Existing automated placement tools, such as ACCUPLACER, often lack transparency and struggle to identify nuanced linguistic features necessary for accurate skill-level classification. By integrating human-annotated linguistic features, this study aims to contribute to equitable and transparent placement systems that better address students’ academic needs, reducing misplacements and their associated costs. For this study, a 300-essay corpus was compiled and manually annotated with a refined set of 11 DevEdspecific (DES) features, alongside 328 linguistic features automatically extracted from CTAP and 106 via COH-METRIX. Supervised ML algorithms were used to compare ACCUPLACER-generated classifications with human ratings, assessing classification accuracy and identifying predictive features. This analysis revealed gaps in ACCUPLACER’s classification capabilities. Experimental results showed that models incorporating DES features improved classification accuracy, with Na¨ıve Bayes (NB) and Support Vector Machine (SVM) achieving scores up to 80%. The refined features presented and methodology offer actionable insights for faculty and institutions, potentially contributing to more effective DevEd course placements and targeted instructional interventions.application/pdfengRefining english writing proficiency assessment and placement in developmental education using NLP tools and machine learningPersonalDa Corte, MiguelDSpacehttp://dspace.org/items/4a524eae-b359-47fa-8978-028ac5ffb57eDSpacehttp://dspace.org/items/4a524eae-b359-47fa-8978-028ac5ffb57eDa CorteMiguelORCIDhttp://orcid.org0000-0001-8782-8377PersonalBaptista, JorgeDSpacehttp://dspace.org/items/e817fa28-a005-40e2-9ba4-03fdaedd7df3DSpacehttp://dspace.org/items/e817fa28-a005-40e2-9ba4-03fdaedd7df3BaptistaJorgeCiência IDhttps://www.ciencia-id.pt7010-5366-22C5ORCIDhttp://orcid.org0000-0003-4603-4364Researcher IDhttps://www.researcherid.comH-7699-2013Scopus Author IDhttps://www.scopus.com14035269500HostingInstitutionOrganizationalSapientiae-mailmailto:repositorio@ualg.ptrepositorio@ualg.ptISBNIsPartOf978-989-758-746-7DOIIsPartOf10.5220/00133515000039322026-06-02T16:14:56Z20252025-01-01T00:00:00ZHandlehttp://hdl.handle.net/10400.1/29074http://purl.org/coar/access_right/c_abf2open accessDevelopmental education (DevEd)Automatic writing assessment systemsNatural language processing (NLP)Machine-learning models506562 bytesFundação para a Ciência e a TecnologiaInstituto de Engenharia de Sistemas e Computadores, Investigação e Desenvolvimento em Lisboainfo:eu-repo/grantAgreement/FCT/6817 - DCRRNI ID/UIDB%2F50021%2F2020/PTUIDB/50021/20206817 - DCRRNI IDCrossref Funder IDhttp://doi.org/10.13039/501100001871other research producthttp://purl.org/coar/resource_type/c_c94fconference object2025http://creativecommons.org/licenses/by-nc-nd/4.0/http://purl.org/coar/access_right/c_abf2application/pdffulltexthttps://sapientia.ualg.pt/bitstreams/efc14f07-bdf3-4c2d-b76f-04a7f5fca9b6/downloadProceedings of the 17th International Conference on Computer Supported Education (CSEDU 2025)22025 |
| spellingShingle | Refining english writing proficiency assessment and placement in developmental education using NLP tools and machine learning Da Corte, Miguel Developmental education (DevEd) Automatic writing assessment systems Natural language processing (NLP) Machine-learning models |
| status | SINGLETON |
| subject.fl_str_mv | Developmental education (DevEd) Automatic writing assessment systems Natural language processing (NLP) Machine-learning models |
| title | Refining english writing proficiency assessment and placement in developmental education using NLP tools and machine learning |
| title_full | Refining english writing proficiency assessment and placement in developmental education using NLP tools and machine learning |
| title_fullStr | Refining english writing proficiency assessment and placement in developmental education using NLP tools and machine learning |
| title_full_unstemmed | Refining english writing proficiency assessment and placement in developmental education using NLP tools and machine learning |
| title_short | Refining english writing proficiency assessment and placement in developmental education using NLP tools and machine learning |
| title_sort | Refining english writing proficiency assessment and placement in developmental education using NLP tools and machine learning |
| topic | Developmental education (DevEd) Automatic writing assessment systems Natural language processing (NLP) Machine-learning models |
| topic_facet | Developmental education (DevEd) Automatic writing assessment systems Natural language processing (NLP) Machine-learning models |
| url | http://hdl.handle.net/10400.1/29074 |
| visible | 1 |