Available For Work
Local Time ( IST )
Video AI at the Edge: From C++ to WebAssembly and Beyond
Role
Team Lead/Full Stack Engineer
Focus
Innovative R&D Solutions
Technical Leadership
Cross-Platform Engineering
Company
Sensority, startup
Location
Israel
Industry
AI Video analytics, Security, Clearance
Technology
Cloud, Desktop, AI
Intro
As a Team Lead and Full Stack Engineer, I worked on a video analytics product for an Israeli startup that focused on AI-driven analysis of video calls and records. The project involved migrating their BE2G desktop solution to the cloud and delivering a JavaScript SDK for browsers. My role covered C++, WebAssembly, JavaScript, and Golang development, making it one of the most technically diverse projects I’ve handled.
Results
Despite the company closure, the project resulted in the successful development and documentation of a functional SDK. This SDK was shared with integrators, showcasing a robust solution built with cutting-edge technology. The experience provided valuable insights into the challenges of bridging REST and gRPC protocols, as well as hands-on learning with Go and WebAssembly. While the product never went live, the technical solutions implemented remain a significant accomplishment and a testament to the team's ability to overcome challenges with innovative approaches.
1s->17ms
SDK processing time reduction
40%
Integration time reduced
65%+
Comparing to TensorFlow.js, RAM consumption reduced by
⚙️ Key Challenges & Strategic Solutions
1. On Premise -> Cloud First Transition
Challenge:
Migrate the C++-based core solution to the cloud while maintaining performance and modularity.
Solutions:
Collaborated with the Solution Architect to separate the core C++ logic into two components: a server-side component and a customer-facing SDK. This approach enabled us to utilize cloud infrastructure while delivering a modular and robust SDK for customers. Led the research and development of the JavaScript SDK to run in browsers, making our product accessible to the majority of the market.
2. WebAssembly Integration for Browser-Based C++ Logic
Challenge:
Integrate heavy C++ logic into the browser for preprocessing video frames efficiently.
Solutions:
The C++ codebase was extended and compiled to WebAssembly, allowing it to run directly in the browser. This integration ensured that we could perform complex preprocessing tasks at scale without sacrificing performance.
3. Optimizing Browser Performance for 4K Video Processing
Challenge:
Process high-resolution video frames (including 4K) in the browser without dropping frame rate or overwhelming the main thread in near real time.
Solutions:
Architected and developed the JavaScript SDK to utilize three Web Workers, each performing different tasks: capturing frames, extracting polygons and metadata, and preprocessing the extracted data. This approach minimized the load on the main thread and ensured high frame rates, even with 4K video.
4. Enhancing Face Tracking Performance
Challenge:
Achieve High-Performance Face Tracking in the Browser with Web Worker Compatibility
Solutions:
We first tried reusing the same C++ code compiled to WebAssembly, but it was too slow. We also tested TensorFlow.js and OpenCV for JavaScript, but both were underperforming. I then found a faster alternative library and optimized its performance. To make it compatible with Web Workers, I rewrote the library to use an offscreen (or "shadow") canvas, which significantly improved performance and enabled smooth operation within the browser.
5. Exploring WebCodecs for Raw Camera Data Access
Challenge:
Access raw camera data in the browser without browser-applied color correction, which interfered with precise AI-driven color analysis
Solutions:
I explored the WebCodecs API (in its alpha stage at the time), which allowed direct access to the raw camera feed, bypassing the color correction applied by the browser. Although the solution didn’t fully meet the expectations at the time, the research and collaboration with Google engineers provided valuable insights for future advancements in this area.
6. Developing a REST-to-gRPC Proxy for Browser Integration
Challenge:
Enable communication between the browser and a gRPC server, a technology not natively supported by browsers
Solutions:
I developed a REST-to-gRPC proxy in Golang, bridging the gap between the browser's REST calls and the gRPC server. This was my first time working with Golang, but I successfully implemented the solution, adding features like stickiness to ensure a seamless connection between the client and server.
🧠 Technical Leadership
R&D Leadership
Led the R&D team in developing a cutting-edge, browser-based video processing system, pushing the limits of in-browser performance and architecture.
Technology Exploration
Drove exploration of advanced technologies like WebAssembly, Web Workers, offscreen canvas, WebCodecs, TensorFlow.js, and OpenCV.js to identify the best-fit solutions.
Performance Optimization
Oversaw system-wide optimizations, ensuring smooth handling of high-resolution video (including 4K) while minimizing main-thread load.
Strategic Adaptation
Guided the team in making smart technical pivots based on hands-on experiments and performance data, aligning innovation with product needs.
Mentorship & Collaboration
Mentored team members, fostered a culture of experimentation, and encouraged cross-functional collaboration to tackle complex technical challenges.
Stack
Conclusion
Although the company ultimately closed, this project resulted in a highly efficient and technically sophisticated SDK that successfully bridged browser-based systems with backend gRPC services. The work not only showcased advanced engineering — such as cutting processing time from 1 second to 17 milliseconds — but also delivered meaningful value to integration partners, who were able to evaluate and integrate the solution into their pipelines. This project serves as a testament to the R&D team’s ability to tackle challenging problems, push technical boundaries, and create production-ready solutions with real-world applicability and impact.
@2025, All Rights Reserved
