Evaluation map
Camera field of viewLocal inferenceReview exceptions

Measure the scene before the model

HomeEdge AI for a defined camera task

Intelligent hardware

Edge AI for a defined camera task

Plan the whole site: cameras, compute, network, installation and a measurable acceptance test.

Discuss a site evaluation

This page is for integrators and site owners who need a camera recognition task deployed at a factory, farm, campus or commercial site. Edge AI keeps recognition on site, but whether it works depends on the whole system: cameras, compute, network, installation, storage and operations, not just the compute rating of one board.

CNPS helps organize site requirements, run a representative site test and produce an evaluation brief. Deployment references published by vendors such as Seeed show what is possible, but they are not CNPS deliveries and do not guarantee the same result under your lighting, distances and network conditions.

Where this fits

Safety and compliance monitoring on production lines or in warehouses

Detect missing protective equipment, boundary crossing or unusual dwelling. Evaluate whether the false-alarm rate lets staff trust alerts over time, and how the system performs at night and in backlight.

Continuous observation in farming and agriculture

Count animals or crops, or recognize behavior and condition. Evaluate the long-term effect of dust, humidity and changing light on the cameras and compute units.

Local recognition at network-constrained sites

Remote sites, or sites that cannot send video to the cloud, need local recognition with only results uploaded. Evaluate behavior during disconnection, recovery and local storage capacity.

The expected output

A site requirement and evaluation brief

What to share

Camera types, site layout, lighting, objects to detect, network conditions and the operational owner.

What to evaluate

  • Agree on the detection task and ground truth.
  • Record false alarms, misses and end-to-end latency.
  • Test lighting changes, disconnection and recovery.
  • Include installation, storage and support in the cost.

How the evaluation runs, step by step

  1. Agree the detection task and ground truth
    State in one sentence what to detect, under which conditions and who acts on an alert. Agree how ground truth is produced and which false-alarm and miss levels are acceptable at acceptance.
  2. Complete the site survey and collect representative footage
    Use the edge AI site survey to record camera types, mounting positions, lighting, distances, network and power. Collect footage at different times of day, including the worst conditions.
  3. Test candidate configurations on real footage
    Compare camera and compute combinations on the same footage. Record false alarms, misses and end-to-end latency, and simulate lighting changes, disconnection and device restarts.
  4. Cost the full system and write the evaluation brief
    Include installation, cabling, storage, spares, support and operating staff in the cost per camera and per site. The brief becomes the shared basis for the quotation and acceptance.

Common mistakes

  • Judging site performance from demonstration videos
    Demonstrations are usually filmed under ideal lighting and distance. Your site has to be tested under its worst conditions to know whether the system will keep running.
  • Comparing only model inference time
    End-to-end latency includes capture, transmission, inference and alert delivery. Only the full chain tells you whether an alert arrives in time for someone to act.
  • Leaving out installation and operating costs
    The board price is a small part. Cabling, enclosures, storage, spares and remote maintenance often decide whether a project is sustainable.

What to record during the pilot

Write these down before the test starts, so the decision rests on numbers you collected rather than on impressions.

  • False-alarm rate per camera per day
    Count alerts with no real event, by camera and by time of day. A rate staff can live with for weeks is the real threshold, not the vendor’s demonstration figure.
  • Miss rate against ground truth
    Compare alerts with the verified record of real events. Report misses separately for daytime, night, backlight and the worst weather you collected.
  • End-to-end latency
    Measure from the event in the footage to the alert on the responder’s screen or phone. Record the median and the slowest ten percent.
  • Uptime, disconnection and recovery
    Log every device restart, network loss and how long recognition and uploads took to recover, including whether local storage kept the results.
  • Cost per camera and per site
    Add cameras, compute, cabling, enclosures, storage, spares, installation and support, then divide by the number of cameras and sites you actually plan to run.

Key terms on this page

These terms are used in the same sense on this page, in the linked resource and in our quotations.

Detection task
One sentence stating what to detect, under which conditions and who acts on the result. Everything else on the site is measured against it.
Ground truth
The human-verified record of what actually happened in the footage, used to judge whether each alert was right or wrong.
False alarm and miss
A false alarm is an alert with no real event; a miss is a real event with no alert. Both rates are needed, because tuning one usually moves the other.
End-to-end latency
The time from the event in front of the camera to the alert reaching the person who acts, including capture, transmission, inference and delivery.
Edge device / compute unit
The on-site computer that runs recognition, such as a Jetson-class or NPU board. Its rating matters less than its behavior under heat, dust and power interruptions.
Site survey
The structured record of camera types, mounting positions, lighting, distances, network and power at each location, completed before any equipment is chosen.
Acceptance criteria
The false-alarm, miss and latency levels agreed in advance that decide whether the installation is accepted and paid for.
Cost per camera, per site
The full figure including cameras, compute, cabling, enclosures, storage, spares, installation labor, support and operating staff.

What to confirm

Hardware capability and demonstration videos do not guarantee performance in your environment. A representative site test is needed.

Common questions

Do we have to buy equipment before evaluating?

Usually not. Complete the site survey and footage collection first, then test with a small number of representative devices. Buying in volume before the task and acceptance criteria are clear carries more risk.

Can CNPS supply a ready-made recognition solution?

CNPS provides requirement organization, the site evaluation method and hardware sourcing. Whether a recognition task is feasible depends on your site conditions and must be shown by a representative test, not by applying a template.

What accuracy can we expect?

We do not promise accuracy before testing. False alarms, misses and latency should be measured on your real footage and judged against acceptance criteria agreed in advance.

Can we reuse the cameras we already have?

Often partly. Resolution, lens angle, night performance and stream format all affect recognition. Include existing cameras in the site test and be prepared to replace those that fail the worst-condition footage.

How long does a representative site test take?

Collecting footage across times of day and weather usually takes one to three weeks, and the comparison itself a further week or two. Rushing the footage stage is the most common reason results do not hold after installation.

What happens to the video and the results?

Decide before testing where footage is stored, for how long, who can view it and what leaves the site. Local recognition can limit data transfer, but retention and access rules still need an owner.

Continue exploring

Industry references are not presented as CNPS customer deliveries.

Let’s start with your real-world challenge.

Tell us what you need to improve, where you work and when you want to begin.

Request a quotation for devices or a pilot.