How to use a trained yolov8 object box detection model directly in tensorflow js in pure javascript? [closed]
I'm currently trying to use a trained yolov8 object box detection model (for 13 ish classes) from ultralytics, directly in tensorflow js in pure javascript.
I don't want to use roboflow js, onnx etc for running it ; or any framework like node-js implementation etc to serve it.. I want to have a fully working yolov8n model detecting custom objects in images/videos/webcam direclty on browser using only tensorflow js.
In the purpose of having something quite usefull (more than 10fps in webcam), i know that i have to use a quantized model for it. So my second question : What's also the code needed to use a custom yolov8n quantized model ?
I looked pretty much at hundreds of links, topics etc on the subject but i haven't found any for this particular use case. I'm also open to someone that has the full solution for an ssd-mobilenet custom training/deploying solution in pure tensorflow js for object detection with boxes.
You would be very nice and serving a lot of people i think if you share the full solution with codes on how to do it here !
Ps : I know that it's possible but any examples I'have found yet are either using node-js, or are for object classification etc... I haven't found any for my specific purpose : pure smooth tjfs yolov8 object detection with boxes.
[EDIT] If you're willing to help : Just give us the code to use a custom trained yolov8n on image/webcam using tf.loadGraphModel for a normal/16-quantized/8-quantized to work as a working example for this use-case that is nowhere on the internet.
[EDIT 2] I basically came up with this code as of right now... It seems to correclty pass the image to the model but i don't understand the output. What's the catch here ? :
html
js tf.ready().then(() => {
const modelPath = "https://localhost/test_php/yolov8_model_json";
tf.tidy(() => {
tf.loadGraphModel(modelPath, { fromTFHub: true }).then((model) => {
const mysteryImage = document.getElementById("mystery");
const tfimg = tf.browser.fromPixels(mysteryImage).toInt();
const tfimg_res = tf.image.resizeBilinear(tfimg, [640, 640]);
const expandedimg = tfimg_res.transpose([0, 1, 2]).expandDims();
// const predictions = await model.executeAsync(expandedimg);
const predictions = model.predict(expandedimg);
console.log(predictions);
// get image width, height from output
const imageWidth = predictions.shape[2]
const imageHeight = predictions.shape[1]
console.log('imageWidth:');
console.log(imageWidth);
console.log('imageHeight:');
console.log(imageHeight);
// create a regular array from the model's output prediction (tensor)
const map = Array.from(predictions.dataSync())
console.log('map:');
console.log(map);
// create an image with the segmentation color data
let imageData = new ImageData(imageWidth, imageHeight);
imageData.data.set(map);
let canvas = document.getElementById('canvas')
let ctx = canvas.getContext('2d')
ctx.putImageData(imageData, 640, 640)
console.log('processOutput completed:', imageData)
output : console log
Comments
Post a Comment