Category Archives: Hardware Verification

Portable Stimulus Specification (PSS) and the Reuse Revolution

The enclosed technical article is a collaboration between an expert user, Mike Bartley (Test and Verification Solutions), with a senior solution architect, Sharon Rosenberg (Cadence) to provide a complete picture of the problem statement, a solution paradigm and technology, and the user benefits and impact of applying PSS technology.

The Challenge

As products become more complex and market windows continue to shrink, efficiency in product development can translate directly into competitive advantage.  We understand what improves efficiency in verification: abstraction and reuse; but how to achieve that?

Accellera are looking to address this through the Portable Stimulus Specification (PSS).
The reuse revolution started in design IP and has contributed hugely to design efficiency. Verification IP soon followed in the form of eVC and latterly UVC but these miss the main point: the currency of verification is stimulus (and checkers, of course).  Also, verification has to deal with multiple dimensions of reuse:

  • Hierarchy (block, subsystem, SoC, system)
  • Platform (simulation, emulation, FPGA prototype, Silicon)
  • Project (reuse from one project to the next)
  • PSS tries to deal with all these through abstraction.

Note the picture below is a good start to this but misses Silicon and the project dimension)

Figure 1: the need for portability

Read the Full Article

  • Click here to Read the Full Article (pdf)

    For demonstrating the PSS capabilities lets introduce a known and common industry challenge – the cache and IO coherency challenge.  For optimization, both CPU-cores keep copies of the memory in close-by fast internal caches.  If a CPU core cached and updated its memory block copy, it is key to ensure that no other core updates the common memory, resulting and incoherent cache.  In order to maintain coherency, sub-systems must communicate between them (e.g. using snoops) to ensure no device is invalidating an already modified cache.  Achieving such scenarios requires highly-parallel SW driven tests which can target both the inter-cluster intra-cluster, and IO traffic.

    Figure 2: SOC coherency traffic

    The diagram above shows a representative system in which the well synchronized traffic (the red arrows), comes from the CPU cores, and peripherals to stress the system coherency mechanism. SoC designs often have 3rd party IP for their CPUs, GPUs, DSPs, and IO devices, but cache coherent interconnects involve custom, often complex integration.

    The PSS Solution

    To better understand how PSS solves coherency challenges, let us discuss PSS usage flow and concepts.  As regards flow, in the PSS use model the user uses a two-step approach. In the first step, the user captures the use case behaviors in the form of actions and their combination rules / dependencies.  In the second step, the user leverages the actions of the first step to define the desired use cases using PSS activities.  The activities need not be complete; they can be partially specified. A PSS compatible tool can leverage the action’s combination rules or dependencies to do any of the following:

    • validate the correctness of the user activity request
    • Constraint randomization for legal attribute value assignment
    • Infer and add missing actions
    • Determine the legal scheduling given the available system resources.

    Figure 3: PSS Usage Flow Diagram

    PSS introduces several powerful concepts that can be combined to address multiple industry challenges.  The figures below illustrate PSS actions that can serve many teams and use-cases on different execution platforms.

    Figure 4: CPU or TB Bus Transactor Actions

    Note that the same write_data action can be implemented by a master on the BUS or via CPU core to write the memory buffer. At the same time, the read_check_data action requires a memory buffer as an input in order to be executed.  This is a dependency of the read_check_data.  Let us now see an example of a scenario that can be achieved with this minimal vocabulary of three actions that we have just created.  Requesting a PSS tool to executed a read_check_data action as a desired scenario will result in the following solution:

    Figure 5: PSS Complete Solution For a read_check_data Scenario Request

    Using the action model and combinations rule, a PSS tool inserted the write_data action to be executed before the read_check_data.  It will randomly assign processors or a master to execute the write and read actions and insert sync points between them to make sure that the reading begins only once the initialization is done.  PSS allows the requesting of sophisticated use cases and scenarios using compound actions. Compound actions have an activity block in which users can request more sophisticated, procedure-like flows.  For example, to stress and interconnect, the user can request three initializations, followed by multiple legal copies and a read_check_data at the end, and get something like this:

    Figure 6: Parallel Copy Chains (Timing and Data-Passing Diagram)

    In the above example, action types are shown in different colors to make it easy to see that we start with three initialization write actions (in gray), followed by multiple copy actions (in cyan) and finishing with check actions (in yellow), exactly as requested. The memory buffer and the attributes are colored in pink. The participating processors and the accessed memory locations are all carefully selected to enable this highly parallel scenario.

    But how are these three actions relevant to cache and IO coherency?

    PSS allows layering constraints from above at both the actions and the flow-objects (in this case the memory buffers). This means that I can constrain the memory buffer location to be in a specific cache region, and stress the cache mechanisms by forcing the CPU cores to snoop and invalidate other CPU cache lines. While extremely powerful, this is not enough for coherency verification. PSS allows self-checking that user can implement in both front-door (the CPUs are checking the result), or back-door (a backdoor task wakes-up at the right time to perform the check). If all the processors randomly write in an uncontrolled manner to the same location, how can the testbench predict the expected result?

    There are two strategies to overcome the self-checking challenge. The first one is called “false sharing” and the second one is “true-sharing.”  In the false sharing mode, though all the cores are writing at the same time to the same lines, each CPU core is writing to its own dedicated bytes, so that there are no actually simultaneous writes to the same bytes. The figure below shows a false sharing scenario:

    Figure 7: Coherency False-sharing Scenario (Timing-only Diagram)

    In the diagram, you can see the execution steps, where the model

    • Selects a valid cache region,
    • Loads the memory to the participating CPU caches,
    • Performs multiple parallel reads and writes, and
    • Finalizes with a self-check at the end.

    Note that this screen shot is showing only the timing information of the scenario without the flow-objects that are passed between the actions. Here the cores do reads and write operations without any synchronization between them.

    The true-sharing coherency strategy allows the CPU cores to write the same memory addresses (hence true sharing). To allow checking, the tool provides the needed sync point to predict the expected result.

    Figure 8: Coherency True-sharing Scenario (Timing-only Diagram)

    In the true-sharing scenario above, after loading the memory buffer into the CPU-caches (cyan), the activity between the CPU cores are well synchronized (green). This allows prediction of the expected result (pink). But what if I need to connect my I/O device to participate in the coherency scenario. For example, the PCIE with its own SMMu might write to a cacheable region. Will my system support this scenario?

    This is where the PSS concepts are extremely useful. Just like any other action, the Root complex and Endpoint devices can read and write memory buffers. Modeling the PCIE devices in PSS provides seamless integration with coherency scenarios.

    Figure 9: Cache and I/O Coherency with the ARM and PCIE actions

    Packaging Actions for Reuse

    True-sharing and false-sharing are just two examples of actions that could be packaged into a reusable library.  For example, Cadence provides SOC action libraries that allow coherency, low-power, memory virtualization, connectivity, and performance scenarios. Some of the actions are generic for any architecture, and some are tuned for ARM architectures, generating combinations of highly-parallel C and assembly tests.  Cadence also provides a PCIE library that can validate the integration of the PCIE controller and could be mixed with the SOC actions to achieve other interesting SOC scenarios.  All libraries can be configured for specific project needs and be mixed with other user-defined actions to efficiently achieve project specific scenarios.  The PSS action dependencies and combination rules allow the PSS tool to produce legal SOC scenarios for both SW driven and host driven (transactional) configurations.


    The most noticeable impact of using PSS is the automation and organized process that this approach injects into SOC projects. Test creation in SOC environments is often manual work that can be both error-prone and time consuming following an iterative development process. PSS allows harnessing technology, such as Perspec, for automating the task of test generation. The tests are correct-by-construction as the action model ensures legal completion of the multiple SOC dependencies. Users can use functional coverage to achieve a well-defined process of setting a goal and ensure timely convergence of the verification process. PSS allows for projects that started with a compromised test plan that was limited by their test-creation capabilities transition to a more desired, thorough plan when they adopted the automated approach. Alternatively, users can decide to reduce the overall verification time to achieve the same level of thoroughness. Many times, a library provides more use-cases that the user initially aimed for, including coverage, a verification plan that can be adapted according to needs and self-checking tests without writing a single PSS line.

    The other value to be mentioned is the portability and reuse aspect.  As mentioned at the start of this blog, verification has to deal with multiple dimensions of reuse: Hierarchy; Platform; and Project (reuse from one project to the next).  PSS tries to deal with all these through abstraction.  We have discussed this in the context of memory coherency where multiple CPU-cores and peripheral devices are sharing memory whilst also keeping locally cached copies for performance.  Without a library, our first step is to define some core memory-related actions (such as write, read-and-check) with rules for combining and synchronizing them.  We can now define the CPU cores and peripherals onto which those actions can be mapped as well as adding constraints (e.g. defining memory regions to force true data sharing).  Synchronization allows us to know the actual ordering of reads and writes and hence to know the expected results for self-checking tests. The actions and produced scenarios can be mapped to either transactions or software making them portable across multiple simulation and emulation environments.  The software tests can also be ported to silicon. Thus, PSS has allowed us to define truly portable test through abstraction.

    How Can T&VS Help?

    As a leading expert in test and verification and having worked on numerous test and stimulus projects it is natural that T&VS have been closely tracking the development of PSS since its inception and have worked closely with many of the companies on the Accellera Portable Stimulus Working Group. With the growing availability of tools supporting PSS T&VS are now ideally placed to provide expert, independent advice to our clients on the use of PSS and to undertake verification projects based on the growing number of tools that support the standard.

    • Discover how T&VS can help deliver your next PSS -based test and stimulus project. Read more


T&VS Bring V&V Expertise to Innovate UK Funded Autonomous Commerical Vehicles Project

Dubbed “RoboPilot”, the project will see Charge Automotive lead a consortium, including T&VS, to bring autonomous racing technology to the light commercial vehicle market and demonstrate SAE level 4 autonomy.

RoboPilot features in the second stream of Connected and Autonomous Vehicle (CAV2) projects that were recently awarded £31m of Innovate UK led funding.  The CAV2 projects are focused in the areas of vehicle energy reduction and air quality improvements.

T&VS will work to accelerate the ‘safety’ components of the consortiums vision, working with partners on the verification and validation of the complex cyber physical systems involved in autonomous vehicle deployment.

RoboPilot combines input from sensors around the vehicle such as radars, cameras, ultrasonics and lidars (light sensors to measure the distance to a target object) with mapping, artificial intelligence and fleet information, which is then acted on by autonomous software.

The project will initially develop & demonstrate autonomous driving functionality for an electric delivery van, which can then be adapted and rolled out to larger trucks and buses.

Mike Bartley, CEO and Founder of T&VS said, “It is great to have our expertise in the verification and validation of advanced driverless technology recognised by Innovate UK.  We look forward to working with our consortium partners to spearhead the development of the UK’s world-leading autonomous vehicle technology.”

Read the coverage from Freight in the City

Consortium Members

  • Charge Automotive Ltd (Lead)
  • AXA UK plc
  • Loughborough University
  • University of West of England
  • University of Bristol
  • UPS UK Ltd
  • South Gloucestershire Council
  • Test and Verification Solutions Ltd
  • Thales UK Limited

About CAV

The Centre for Connected and Autonomous Vehicles (CCAV) is a joint unit of the department for Business, Energy and Industrial Strategy and the Department for Transport. CCAV is a single point of contact for those in industry, academia and internationally set up to keep the UK at the forefront of the development of connected and autonomous vehicle technology.

Find Out More

If you would be interested in applying the lessons from this project to your business please contact one of our V&V consultants today.

Alternatively call one of our local sales offices.


T&VS Contributes to PEnDAR Project

A Feasibility Study Into the Cost-Effective Capture, Validation and Verification of the Performance (and Resource Cost) of Complex Systems During the Design Process

PEnDAR (Performance ENsurance by Design, Analysing Requirements) is an InnovateUK funded collaborative project between Test and Verification Systems (T&VS), Predictable Network Solutions (Lead Participant), and Vodafone Group.

With society’s increasing dependence on Information and Communications Technology (ICT) the need for better means to predict and assure the performance of critical infrastructure grows.  Today, the performance of large-scale Cyber Physical Systems and System of Systems is often an unplanned emergent property that can vary substantially during operational lifetime.  Although this hazard is sometimes validated as part of system commissioning, it often finds its way into deployed systems, impinging on the systems usefulness as well as increasing the total lifetime cost.

The PEnDAR project is a study into the feasibility of systematically considering both performance and resource costs early in the system development lifecycle (SDLC). The goal is to consider how to enable the Validation & Verification of cost and performance in the field of distributed and hierarchical systems via sophisticated but easy-to-use tools.

It is thought likely that this approach will be applicable to both new and established systems and be able to support both initial and ongoing incremental development.

This feasibility study aims to investigate the technical issues involved in effectively incorporating the mature mathematical techniques already available to capture, validate and verify the performance (and resource cost) of such complex systems during the design process rather than as an emergent property from the design process.

T&VS Contribution

As part of this collaborative project T&VS investigated the following critical items and incorporated the results into the final project report.

  • The feasibility of integrating performance V&V into a safety standards-compliant process suitable for safety-critical applications such as automotive.
  • Extending existing standards-compliant requirement sign-off tools to manage the capture and decomposition of performance/resource V&V requirements into specifications, features and sign-off criteria.
  • How the performance V&V process can be incorporated into a standards-compliant workflow for safety-critical applications.
  • Assessed the potential savings of performance/resource V&V in automotive software development and the potential benefits of applying a similar methodology in the SoS integration market.

Deliverables – Slideshare & Recordings

The findings from the study are planned to be presented as a paper in a special issue of IEEE Design & Test.

Find Out More

If you would be interested in applying the lessons from this project to your business please contact one of our V&V consultants today.

Alternatively call one of our local sales offices.


Safety Critical Verification – Who Can Help with my IC Design?

In a recent post on the EDA article clearinghouse,  Jim Hogan of Vista Ventures LLC looked into how all the EDA vendors  and leading consultants rate in terms of support for Safety Critical IC Verification.

Jim starts by stating:

“Today there is no one EDA company that has all the tools necessary to do safety critical chip verification (SCV), though virtually all are rushing to expand in this space.  Rarely does a week go by where we don’t see yet another announcement around safety critical verification.”

In the article he looks at all the key vendors; Cadence, Mentor, OneSpin, Synopsys and Aldec, and provides a matrix of their verification tools they offer before expanding on this to provide additional analysis of their solutions and place in the market.

In addition he looks at the leading consultancy companies that can provide support for safety critical IC design, and commenting on T&VS states:

“Mike Bartley has grown his consulting practice to help companies with their safety-critical methodologies.  It is mostly focused in the systematic area over random, but he does both.  T&VS has both a methodology service as well as a tool for verification
planning called asureSIGN.  asureSIGN lets you merge information from multiple sources; for example, you can merge your simulation data with OneSpin’s formal verification results using asureSign.”

This update forms the first of a number of recent posts from Jim Hogan covering safety critical IC design and Formal Verification:

  1. Using Formal along with random fault verification
  2. ISO 26262 certification and systematic verification
  3. How Safety Critical Verification is Next Big Thing
  4. Rating all the EDA vendors on Safety Critical IC design

About DeepChip is a 20 year old clearinghouse where semiconductor chip  designers contribute data-intensive papers and articles of first-hand  evaluations and production  benchmarks of commercial EDA tools.

About Jim Hogan

Jim is currently the managing partner of Vista Ventures, LLC. Jim has worked in the semiconductor design and manufacturing industry for more than 40 years gaining experience as a senior executive and board director in electronic design automation, intellectual property, semiconductor equipment, material science and IT companies.  Currently, Jim serves on several private companies’ board of directors.  Additionally, Jim serves as a strategic advisor to several private and public companies.

How to Protect the IoT with secure hardware

As the IoT devices continues to expand exponentially, security threats to hardware is an also growing concern and it becomes more of a reality to the organizations, the importance of securing the billions of remote, connected objects, networks against cyber-attack becomes increasingly challenging. This article explores why the use of secure hardware is recommended for protecting today’s user-accessible networked IoT infrastructure.

Read More

Learn how T&VS IoT Security services allow you to take a comprehensive approach to maintain the security, and protect your IoT devices from cyber threats.

What are your Biggest V&V Challenges?

Verification Futures has been running since 2011 in the UK and has also run in France, Germany and India.  During that time over 30 verification managers and engineers have presented their 3 top verification challenges, resulting in close to 100 different challenges being shared with the community (see table below).

Analysing that data tells us that their biggest challenge is integrating different languages, tools and strategies.  Innovation in hardware verification has brought us a slew of verification technologies to help us verify increasingly complex designs.  However, understanding how to select the right strategy combining methods, languages and tools, and then combining the results continues to be the biggest challenge we face.

As mentioned, design complexity continues to increase with a consequent increase in verification complexity.  This is cited by 8 of the speakers making it the second biggest challenge closely followed by completeness, mixed signal and debug.  Verification managers see closing verification as a major challenge leading to potentially long tails in the project execution.  Mixed signal verification challenges have been on the increase as speakers see more analog blocks appearing in designs.

Debug continues to appear very high in the list and industry research also puts debug as taking the largest percentage of verification effort on our projects. To my amazement most debug is still performed with waveforms and “printf” (although UVM allows us to write “printf” in more sophisticated ways 🙂 )  Surely debug is ripe for a major innovation to help us all save time for our scarce verification resource.

The table below gives the summary of the hardware verification challenges presented over the past 6 years.  This year the conference includes more software testing, with tracks on both safety and security.  The challenges are coming from Qualcomm, Thales and GE Aviation and as expected there are challenges around qualifying safety-related products but also familiar ones shared between both hardware and software; namely dealing with complexity and predictable execution of verification plans.

Join Us for Verification Futures 2017

Why not join us for Verification Futures this year on 6th April 2017 to hear more about the challenges and the solutions being offered. It is free to attend and there is both physical attendance in Reading, UK or virtual attendance available.

What Are Your Biggest V&V Challenges?

Help us to discover the top challenges that the community is currently facing in performing efficient hardware and software V&V by completing a short survey.  It has just two questions and shouldn’t take more than 2 minutes to complete.  Results will be presented at VF2017.  Thank You.

Table of Challenges

Challenge Description # Challenges
Integrating Languages, Tool and Techniques 10
Complexity 8
Completeness 7
Mixed Signal 7
Debug 7
Productivity 5
Requirements Driven Verif/ISO 26262 5
Reuse 5
Resources 5
FPGA Specific 5
Scalability 4
System 4
Integration Verification 3
Power Verification 3
EDA tool Integration 3
Design for Verif 2
Demonstrating Bug Absence 2
Synthesis/Timing Constraints 2
Performance 1
Change 1
Leading Edge Technology 1
Verification Data Mgt 1
Predictability 1
Measuring Test Bench Quality 1
IO Muxing at SoC Level 1
Lack of Models for Verification 1

NMI Software Validation & Verification Event – November 10, 2015

nmi-parent-logo-complete-loT&VS have supported the NMI in the development of a series of events to help companies better understand and overcome the challenges faced in the validation and verification (V&V) of complex electronic systems.

The first of these events will focus on the V&V of Software where the complexity of the environment and the potential for unexpected emergent behaviors means that traditional approaches are not sufficient.

This full day event will look in depth at how software across multiple sectors (automotive, avionics, medical, etc.) can still be made compliant to the appropriate safety standards, and secure from external threats given a systems likely connectivity.

T&VS Presentation

Mike Bartley Founder & CEO, TVSAt the event, T&VS will presenting in the “State of the Art” track on the following:

What can we Learn from Other Disciplines
Mike Bartley, CEO and Founder, T&VS

Event Summary

  • NMI Software Validation & Verification
  • November 10, 2015. — 9:00am – 5:00pm
  • UWE (University of the West of England), Bristol
  • Additional Event Information

Provisional Agenda

Please check the NMI Event Website for the very latest agenda.

  • Welcome & Introduction
    Dr. Derek Boyd, Chief Executive, NMI
  • Keynote Presentation
    Jack Cunningham, Director of Software Engineering, Thales UK
  • Software Verification and Validation Competition (see below)
    Robin Kennedy – Knowledge Transfer Manager, KTN
  • Challenges on Software V&V
    Rob Burton, Head of Software Engineering, Raytheon UK
  • Validation of Mars Missions
    Roger Ward, Technology Manager – Space Division, SCISYS UK
  • Case Studies
  • Disrupting the Smart Cities Market
    Wolfgang Bruchner, CTO, Cascoda
    Sean Redmond, Founder, Vertizan
  • Validating Requirements for the Automotive Industry using Model-Based Systems Engineering (MBSE)
    Professor Jon Holt, Director, Scarecrow Consultants
  • Networking Lunch
  • State-of-the-Art Solutions
  • What can we Learn from Other Disciplines
    Mike Bartley, Managing Director, T&VS
  • Model-based Test Generation for Coverage-Driven Verification of Robotic Code
    Kirsten Eder, Reader, University of Bristol
  • Contestor Verification Approach
    Jamie Hutcheson, Principal Consultant, Altran
  • Verification of Avionics Software to Meet DO-178B/C. What, Why and How?
    Dr. Guillem Bernat, Rapita
  • Verification Techniques for Better Code and Higher Productivity
    Mark Richardson, Field Applications Engineer, LDRA
  • Networking Break
  • Group Discussion on Main Challenges & Potential Solutions
  • Event Close