Internlm-math: Open math large language models toward verifiable reasoning
The math abilities of large language models can represent their abstract reasoning ability. In
this paper, we introduce and open-source our math reasoning LLMs InternLM-Math which …
this paper, we introduce and open-source our math reasoning LLMs InternLM-Math which …
Extend model merging from fine-tuned to pre-trained large language models via weight disentanglement
… The Medium tier includes weights from the 1/3 mark to the 2/3 mark, and the High tier contains
weights from the 2/3 mark to the end. Table 6 quantitatively illustrates the adjustments of …
weights from the 2/3 mark to the end. Table 6 quantitatively illustrates the adjustments of …
Beyond accuracy: Evaluating self-consistency of code large language models with identitychain
… Assume that given NL and PL, there exists a semantics space D (we don’t assume any
specific definition of D) st an interpretation function sem is well-defined as the following: …
specific definition of D) st an interpretation function sem is well-defined as the following: …
Token Alignment via Character Matching for Subword Completion
Generative models, widely utilized in various applications, can often struggle with prompts
corresponding to partial tokens. This struggle stems from tokenization, where partial tokens …
corresponding to partial tokens. This struggle stems from tokenization, where partial tokens …
MELoRA: Mini-Ensemble Low-Rank Adapters for Parameter-Efficient Fine-Tuning
Parameter-efficient fine-tuning (PEFT) is a popular method for tailoring pre-trained large
language models (LLMs), especially as the models’ scale and the diversity of tasks increase. …
language models (LLMs), especially as the models’ scale and the diversity of tasks increase. …
AutoCV: Empowering Reasoning with Automated Process Labeling via Confidence Variation
In this work, we propose a novel method named \textbf{Auto}mated Process Labeling via \textbf{C}onfidence
\textbf{V}ariation (\textbf{\textsc{AutoCV}}) to enhance the reasoning …
\textbf{V}ariation (\textbf{\textsc{AutoCV}}) to enhance the reasoning …
How Do Your Code LLMs Perform? Empowering Code Instruction Tuning with High-Quality Data
Recently, there has been a growing interest in studying how to construct better code instruction
tuning data. However, we observe Code models trained with these datasets exhibit high …
tuning data. However, we observe Code models trained with these datasets exhibit high …
Toolsword: Unveiling safety issues of large language models in tool learning across three stages
Tool learning is widely acknowledged as a foundational approach or deploying large
language models (LLMs) in real-world scenarios. While current research primarily emphasizes …
language models (LLMs) in real-world scenarios. While current research primarily emphasizes …
Pecc: Problem extraction and coding challenges
… The next step involves the model generating executable Python code (D) based on the
provided … In alignment with existing benchmarks, we employ a zero-shot format for PECC, but all …
provided … In alignment with existing benchmarks, we employ a zero-shot format for PECC, but all …
Fast State Restoration in LLM Serving with HCache
… We introduce a storage format designed for fast restoration. Since we adopt a layer-wise
approach to restore the states for the history context, the hidden states of all tokens from the …
approach to restore the states for the history context, the hidden states of all tokens from the …