nemo-evaluator-sdk
Evaluates LLMs across 100+ benchmarks from 18+ harnesses (MMLU, HumanEval, GSM8K, safety, VLM) with multi-backend execution. Use when needing scalable evaluation on local Docker, Slurm HPC, or cloud platforms. NVIDIA's enterprise-grade platform with container-first architecture for reproducible benchmarking.
68
82%
Does it follow best practices?
Run evals on this skill
Adds up to 20 points to the overall score
View guide
Critical
Do not install without reviewing
Security
2 findings: 1 critical severity, 1 high severity. Installing this skill is not recommended: please review these findings carefully if you do intend to do so.
E006: Malicious code pattern detected in skill scripts.
What this means
Detected high-risk code patterns in the skill content — including its prompts, tool definitions, and resources — such as data exfiltration, backdoors, remote code execution, credential theft, system compromise, supply chain attacks, and obfuscation techniques.
Why it was flagged
The documentation describes features that allow arbitrary code loading (custom interceptors/containers), logging/saving of full requests/responses, and reporting progress to arbitrary URLs — capabilities that can be (and easily abused to be) used for credential theft and data exfiltration or to introduce remote code execution/backdoors if untrusted modules or containers are used.
W007: Insecure credential handling detected in skill instructions.
What this means
The skill handles credentials insecurely by requiring the agent to include secret values verbatim in its generated output. This exposes credentials in the agent’s context and conversation history, creating a risk of data exfiltration.
Why it was flagged
The skill contains examples that embed an API key literal (export NGC_API_KEY=nvapi-your-key-here and api_key="nvapi-your-key-here"), which would require or encourage the LLM to include real secret values verbatim in generated commands/code, creating an exfiltration risk.
- Repository
- synthetic-sciences/openscience
- Commit
52845c3
- Audited
- Security analysis
Is this your skill?
If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.