Hello, false positives do happen and they often look ridiculous. You can try adjusting the sensitivity if they happen too often. Or adjust the consts file if it’s a particular species/location. It’s true that the birds species in the TensorFlow model are North American. We will offer European species eventually…
An important thing to consider is that we trained the model using birds that took up about 25% of the image – so fairly large, as if you are looking at a bird feeder. This way the pictures get some decent detail of the birds. If you’re far away and birds are small (for example 10% of the image or less) it will not detect (at all, unfortunately.) The state of the art for these kinds of things isn’t perfect… you make trade-offs and assumptions, but improvements are coming along at a fast pace for this kind of deep-learning detection.
Hello,
It’s absolutely possible to train a network on European species. The training for North American species was done using the TensorFlow training utilities. Having heard it described, I don’t think it’s something that can be done in an afternoon, unless you were sufficiently familiar with TensorFlow.
Understood … still it would be great to share where the model came from and references to test data.
BTW one of my goals here is to take the birdfeeder as an example and create a more generic garden wildlife alert camera. First step will be to be able to build, train and deploy myself the existing app … then I can iterate.
Hello,
The training set was obtained from the iNat competition. We combined recent years’ data and trained a network by extracting pictures of the target species. These repositories are an amazing resource We did the training using an AWS server with several CUDA GPUs. You can do this yourself without a fancy GPU, but it can take a long time.
There are several howto guides on how to train your own Tensorflow network. This is a rapidly changing space and if the guide is over a year old, it’s probably already dated. We need to update our models to take advantage of the latest developments… this will happen over the coming months.
Hello,
I believe you need object detection instead of image classification. Here are the latest object detection models. You start with a pretrained classifier (e.g. coco) and retrain it using their script. Since we trained the birdfeeder classifier using mobilenet_v2, there have been several new models added to the mobile models that would be worthwhile giving a try. The birdfeeder has a unique accuracy vs speed requirement. A tensorflow lite model would be too inaccurate, while a resnet model would be too slow(?)
Fortunately, the iNat database has pictures with classifications and bounding boxes. Many of the pictures in the database are not good for training though, for example, a bird on a power line taken from 100 meters.
The original code that we used is on an AWS server somewhere. I’m putting it on my todo list to try to dig up. We will be doing some training fairly soon (weeks?) using special GPU hardware that we purchased. We’ll provide notes/howto guide when this happens, but happy to answer any questions in the meantime. It’s definitely an exercise for the less faint of heart, but very doable.
The first issue I hit is that the iNaturalist images at https://github.com/visipedia/inat_comp/tree/master/2021 doesn’t contain bounding box data ( older sets do for some birds … but I didn’t find anything for the UK bird set ). So I’m slowly going through and creating bounding boxes.
So far, the training I’ve tried on my imac (with GPU) hasn’t gone that well … the total loss rises to large values and after a few hours the training seems to stop making progress.
Another issue I’ve found is the current tensorflow version, 2.4.3, doesn’t support the frozen inference graph that vizy is using … ie vizy is using the older tensorflow v1 release.
Hello,
We will be digging into this over the coming months. I’m not sure what version of Tensorflow we’ll end up using, but it will certainly be a more advanced version than what’s installed now. We will target a version of Tensorflow that works best on the Raspberry Pi4 (best speed, accuracy, and reliability) whatever version that is.
Are you interested in using Tensorflow Lite? Please share what you’re trying to accomplish so we can feed this info into our effort.
No idea … previously you mentioned that it was too inaccurate.
Well, firstly I thought I would reproduce how you trained the birdfeeder. Then move on to extending it with UK birds and animals so its useful to me in my garden. Tracking at https://github.com/plord12/vizy-training.
Unhandled exception:
Traceback (most recent call last):
File "/home/pi/vizy/apps/birdfeeder/main.py", line 351, in <module>
bf = Birdfeeder()
File "/home/pi/vizy/apps/birdfeeder/main.py", line 176, in __init__
self.tflow.open()
File "/usr/local/lib/python3.7/dist-packages/kritter/tf/tfdetector.py", line 46, in open
self.image_tensor = detection_graph.get_tensor_by_name('image_tensor:0')
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/framework/ops.py", line 3644, in get_tensor_by_name
return self.as_graph_element(name, allow_tensor=True, allow_operation=False)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/framework/ops.py", line 3468, in as_graph_element
return self._as_graph_element_locked(obj, allow_tensor, allow_operation)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/framework/ops.py", line 3510, in _as_graph_element_locked
"graph." % (repr(name), repr(op_name)))
KeyError: "The name 'image_tensor:0' refers to a Tensor which does not exist. The operation, 'image_tensor', does not exist in the graph."
Hi Peter,
Sorry, we aren’t able to get it to run on this end (we tried various ways to grab the tensors.) Are you able to run the graph on your training machine?
Yes. The picture above was generated once trained.
( if it matters I’ve been using tensorflow 2 ).
As mentioned originally in this thread it would be great to understand exactly how the birdfeeder app was generated so we can repeat with a different data set.
The Tensorflow work we did was at the beginning of the project and needs to be updated. Getting the inference code to work with TensorFlow2 shouldn’t be too involved. It requires updating the code in /usr/local/lib/python3.7/dist-packages/kritter-0.0.3-py3.7.egg/kritter/tf/tfdetector.py. – it’s certainly less effort than training your own network
We don’t have any useful notes we can share regarding how the birdfeeder network was generated, unfortunately. We will provide this after we update the network. Our software team hopes to tackle this next month.
All I want to get to is the birdfeeder app working for local (UK) birds. Happy to invest more time and effort on this ( I’ll have more time soon ), but I want to build on what you’ve built rather than re-invent.
Hi Peter,
In short, no we haven’t made headway on this. We’re pushing for the Motionscope application release, which is pushing things back. My apologies…
We did the training using an AWS server with several CUDA GPUs. You can do this yourself without a fancy GPU, but it can take a long time.