www.icact.org

Member Login

TACT Journal Homepage

Call for Paper

Committee

Find My Paper

Author Homepage

Statistics & History

Welcome Message

Paper Procedure

FAQ

Paper Archives

Outstanding Papers

Program & Proceedings

Presentation Platform

Hotel & Travel Info

Photo Gallery

Scheduler Login

Seminar

Archives Login

Sponsors

Virual Oral Question and Answer


IEEE/ICACT20240321 Question.2 Questioner: htzustc@mail.ustc.edu.cn 2024-02-06 오후 3:36:45	IEEE/ICACT20240321 Answer.2 Answer by Auhor yanghao30@huawei.com 2024-02-06 오후 3:36:45
In Slide.10 "CoT Prompt Design", how does the step 4 "Combine the step 1-3 ratings to assign an overall quality estimation" works? What if step 1-3 generate different scores seperately?	In steps 1-3, 5 scores were generated to evaluate a knowledge prompt. Token-level scores classified tokens as good, acceptable, or bad. Sentence-level scores classified sentences as high, medium or low quality. Perplexity scores measured how uncertain the model was about the prompt. Step 4 aggregated the token, sentence, and perplexity scores into an overall score that correlated well with the individual metrics but was not just their average. The overall score provided a robust evaluation of the knowledge prompt.

Select Voice