Annotation Statistics / Static View

[Simon]

Hi

I know I'm using Advene for something which it was not exactly designed for, but I still found it to be the best software available for my purpose.

A short introduction so you can hopefully better understand my problem:
I'm a psychologist running an experiment on child/mother-interaction during a specific task. So for every mother + child I get one video on which they are performing a certain task. What I do now is I create an annotation for every spoken sentence which matches the duration of what is being said. I create those annotations in different annotation types (e.g. directive speech, non-directive speech, and so on..). Like this I get data for how the mother and her child interact during their task.

What I need now for further analysis are the statistics for each annotation type (specifically the number of annotations + total duration). I know I can get these for example by right-clicking on the annotation type. But it would be great if I had one static view which gave me all the information in one place.
Unfortunately I'm not much of a programmer and the advene wiki doesn't help me either.

First of all, is this even possible, to create such a view? Second, would it also be possible to perform some basic calculations (like summarizing the total durations of two annotation types) and report them in the view?
If yes, I would really appreciate some help on how to do this!

Thank's a lot!
Simon

[oaubert]

Hello Simon. It is nice to learn that you find Advene the best fitted for your purpose. Indeed, if our primary focus was on feature movie analysis/critics, we also use it for different purposes like video interview analysis. BTW, would you mind providing some more information (maybe with some screenshots) so that I can feature your use on http://advene.org/collaborations.html ?

On your statistics questions: you can use an Advene static view to generate the statistics that you want. The first part (displaying the number of annotations + duration) is relatively easy. The second part (doing the same for a selected set of types) is doable too, with some more work due to the need to define the selected types. Here is a how-to:

- create a new static view (either in the treeview, by right-clicking on the "Static views" item then "Create a new static view", or from the Edit/Create/Static view menu). Give it a meaningful name, and make sure that the type of the view is "HTML template".

- in the displayed view editor, click on the <> icon (above the text editor) to use the XML code editor instead of the WYSIWYG one

- copy/paste the following HTML template:

Code: Select all

<h1>Statistics</h1>
<ul>
<li tal:repeat="at package/annotationTypes">
<em tal:content="at/id">at</em>
: <span tal:content="at/annotations/length">count</span> annotations
- duration : <span tal:define="s python:sum(a.fragment.duration for a in at.annotations)" tal:content="s/formatted">duration</span></li>
</ul>

<h1>Type selection</h1>
<!-- definition of the list of annotation types (specify the list of ids, separated by spaces, between the single quotes)  -->
<span tal:define="list_types python:[ package.get_element_by_id(t) for t in 'plan at_values'.split() ]">
<ul>
<!-- detail information about each type -->
<li tal:repeat="at list_types"><span tal:content="at/id">type</span> : <span tal:content="at/annotations/length">annotations</span> annotations - duration : <span tal:define="s python:sum(a.fragment.duration for a in at.annotations)" tal:content="s/formatted">duration</span></li>
<!-- total duration computation and display -->
<li><strong>Total </strong>: <span tal:content="python:sum(len(at.annotations) for at in list_types)">count</span> annotations - duration : <em tal:define="s python:sum(a.fragment.duration for at in list_types for a in at.annotations)" tal:content="s/formatted">total duration</em></li>
</ul>
</span>


- do not forget to save your package, so that the view definition is saved.

The first part of the template displays statistics for all types, the second part only for a selection of types.

To visualise the rendition of the template, click on the W3 icon in the view editing window. It will validate your changes and load the rendered view in a web browser, displaying the desired information. You can now tweak the template to adapt it to your needs.
Later on, you can access the view rendition by clicking on the W3 icon from the Advene toolbar (main window) and selecting the view name that you entered.

Note: If you are running Advene on linux, you can first embed a web browser widget in the Advene GUI by drag-and-dropping the W3 icon from the Advene toolbar in an Advene view space. It is sometimes more convenient to have all information in the same application window.

[Simon]

Thank you very much for your extensive answer! This was exactly what I needed.

I guess you could feature our project on your collaborations page, but I'll have to check with the responsible people first. Providing meaningful screenshots hovewer will most likely not be possible since all the videos and data we work with are confidential.

May I ask you another question? Unfortunately the programm does not run very smoothly. Especially the timeline often does not load properly while watching the video, especially when continuos scrolling is activated. What confuses me is that sometimes it runs quite well but then other times the program "dies" after watching 5 seconds of film, even when it's the same film with the same annotations. I could not find a pattern that would explain the problem.
Is this a known issue? Is it due to the program being an early version? Or do you think it has something to do with my setup? (I'm running it on a new/fast laptop with Windows 7)

EDIT: Ah and also, in the wiki it says that it is possible to split an annotation from the context menu.. I did not find this option?

[oaubert]

I was not aware of plain crashes with the timeline. I very rarely use continuous scrolling, it may be at fault. It could be related to a lack of responsiveness of the computer, leading to missed events. How many annotations are present in your package ?
If it happens again, please send me a bug report joining the advene.log file. You can find it in the settings folder, which you can find by using Help/Open logfile folder.

For the annotation split, the menuitem in the context menu is displayed only if the current player position is inside the selected annotation. The current time is used as the new bound for splitted annotations. There is also a shortcut (Control-Return), with the same constraint (the player position must be inside the annotation to be split).

[Simon]

Right now I had two crashes, one after the other, while navigating the timeline. I was watching a completely coded video (686 annotations) and followed the annotations in the timeline. Since I don't dare to use continuos scrolling anymore, what I did was to put it to no scrolling and then manually scroll by clickling the little arrow of the scrollbar. This is where the program crashed. If I continuosly click the little arrow two scroll for more than maybe 10 times while the video is running, the entire program crashes. But not allways when I do this, sometimes it works fine.

Here you can see the two log files for each of the sessions where it crashed:

Code: Select all

Advene v. 1.0 release 20111220 (svn 7523) run at 30/05/12 09:12:13 W. Europe Daylight Time on win32
Using gtk pixbufloader:  C:\Program Files (x86)\Advene\lib\gdk-pixbuf-2.0\2.10.0\loaders\loaders.cache
GTK DLLs found in Advene directory
Using localedir C:\Program Files (x86)\Advene\\locale
Using gstreamer to play sounds
mediacontrol: using gstreamer
SVG: Using rsvgoverlay element
gstreamer: gstreamer: using directdrawsink
gstreamer: Reparent 0x340624L
SVG: Using rsvgoverlay element
gstreamer: check player
C:\Program Files (x86)\Advene\library.zip\advene\gui\main.py:1320: GtkWarning: gdkdrawable-win32.c:2013 drawable is not a pixmap or window
Traceback (most recent call last):
File "advene\gui\views\timeline.pyc", line 1959, in create_annotations
File "advene\gui\views\timeline.pyc", line 591, in finalize_callback
File "advene\gui\views\timeline.pyc", line 3022, in update_legend_widget
File "advene\gui\views\timeline.pyc", line 2804, in resize_legend_widget
AttributeError: 'NoneType' object has no attribute 'get_parent'
C:\Program Files (x86)\Advene\library.zip\advene\gui\main.py:1320: GtkWarning: gtk_widget_size_allocate(): attempt to allocate widget with width -5 and height 15
C:\Program Files (x86)\Advene\library.zip\advene\gui\views\timeline.py:1312: GtkWarning: gdkdrawable-win32.c:2013 drawable is not a pixmap or window
C:\Program Files (x86)\Advene\library.zip\advene\gui\main.py:1320: GtkWarning: gtk_widget_size_allocate(): attempt to allocate widget with width -5 and height 18
C:\Program Files (x86)\Advene\library.zip\advene\gui\views\timeline.py:1259: GtkWarning: gdkdrawable-win32.c:2013 drawable is not a pixmap or window
C:\Program Files (x86)\Advene\library.zip\advene\gui\views\timeline.py:2448: GtkWarning: gdkdrawable-win32.c:2013 drawable is not a pixmap or window
C:\Program Files (x86)\Advene\library.zip\advene\gui\main.py:2315: GtkWarning: gdkdrawable-win32.c:2013 drawable is not a pixmap or window
Traceback (most recent call last):
File "advene\gui\main.pyc", line 2290, in update_extension
File "ntpath.pyc", line 198, in basename
File "ntpath.pyc", line 170, in split
File "ntpath.pyc", line 125, in splitdrive
TypeError: 'NoneType' object is not subscriptable
C:\Program Files (x86)\Advene\library.zip\advene\gui\main.py:2315: GtkWarning: Could not find the icon 'gtk-file'. The 'hicolor' theme
was not found either, perhaps you need to install it.
You can get a copy from:
http://icon-theme.freedesktop.org/releases
C:\Program Files (x86)\Advene\library.zip\advene\gui\main.py:2315: GtkWarning: Unable to retrieve the file info for `file:///E:/Simon/COCON-Videos/Advene/4896.csv': Error stating file 'E:\Simon\COCON-Videos\Advene\4896.csv': No such file or directory
C:\Program Files (x86)\Advene\library.zip\advene\gui\main.py:1320: GtkWarning: inner_clipboard_window_procedure: assertion `success' failed


Code: Select all

Advene v. 1.0 release 20111220 (svn 7523) run at 30/05/12 09:44:47 W. Europe Daylight Time on win32
Using gtk pixbufloader:  C:\Program Files (x86)\Advene\lib\gdk-pixbuf-2.0\2.10.0\loaders\loaders.cache
GTK DLLs found in Advene directory
Using localedir C:\Program Files (x86)\Advene\\locale
Using gstreamer to play sounds
mediacontrol: using gstreamer
SVG: Using rsvgoverlay element
gstreamer: gstreamer: using directdrawsink
gstreamer: Reparent 0x40644L
SVG: Using rsvgoverlay element
gstreamer: check player
C:\Program Files (x86)\Advene\library.zip\advene\gui\main.py:1320: GtkWarning: gdkdrawable-win32.c:2013 drawable is not a pixmap or window
sys:1: Warning: cannot register existing type `GstPlaybinSelectorPad'
sys:1: Warning: g_object_new: assertion `G_TYPE_IS_OBJECT (object_type)' failed
sys:1: Warning: invalid (NULL) pointer instance
sys:1: Warning: g_signal_connect_data: assertion `G_TYPE_CHECK_INSTANCE (instance)' failed
sys:1: Warning: g_object_set_data: assertion `G_IS_OBJECT (object)' failed
snapshotter: error when sending event
snapshotter: error when sending event
snapshotter: error when sending event
Traceback (most recent call last):
File "advene\gui\views\timeline.pyc", line 1959, in create_annotations
File "advene\gui\views\timeline.pyc", line 591, in finalize_callback
File "advene\gui\views\timeline.pyc", line 3022, in update_legend_widget
File "advene\gui\views\timeline.pyc", line 2804, in resize_legend_widget
AttributeError: 'NoneType' object has no attribute 'get_parent'
C:\Program Files (x86)\Advene\library.zip\advene\gui\main.py:1320: GtkWarning: gtk_widget_size_allocate(): attempt to allocate widget with width -5 and height 15


Edit: This is the log file for a session where I tried to watch a video with continuos scrolling. The timeline lags the whole time and the prorgram eventually crashes. Here ( https://www.dropbox.com/sh/kvpgj2iv2sn8r0n/_RWfRjUWpk/2012-05-30_1235.swf ) you can see a video of a similar session.

Code: Select all

Advene v. 1.0 release 20111220 (svn 7523) run at 31/05/12 09:28:48 W. Europe Daylight Time on win32
Using gtk pixbufloader:  C:\Program Files (x86)\Advene\lib\gdk-pixbuf-2.0\2.10.0\loaders\loaders.cache
GTK DLLs found in Advene directory
Using localedir C:\Program Files (x86)\Advene\\locale
Using gstreamer to play sounds
mediacontrol: using gstreamer
SVG: Using rsvgoverlay element
gstreamer: gstreamer: using directdrawsink
gstreamer: Reparent 0x3056aL
SVG: Using rsvgoverlay element
gstreamer: check player
C:\Program Files (x86)\Advene\library.zip\advene\gui\main.py:1320: GtkWarning: gdkdrawable-win32.c:2013 drawable is not a pixmap or window
snapshotter: error when sending event
snapshotter: error when sending event
snapshotter: error when sending event
Traceback (most recent call last):
File "advene\gui\views\timeline.pyc", line 1959, in create_annotations
File "advene\gui\views\timeline.pyc", line 591, in finalize_callback
File "advene\gui\views\timeline.pyc", line 3022, in update_legend_widget
File "advene\gui\views\timeline.pyc", line 2804, in resize_legend_widget
AttributeError: 'NoneType' object has no attribute 'get_parent'
C:\Program Files (x86)\Advene\library.zip\advene\gui\main.py:1320: GtkWarning: gtk_widget_size_allocate(): attempt to allocate widget with width -5 and height 15


[oaubert]

Thanks for the detailed report, I will have a look at this. From the video, it looks like the application does not really crash, but rather freezes for about 10 seconds, then resumes. Can you confirm this? Did you witness any hard crashes (that closed the application and make you lose data)?

[Simon]

Thank you!
Yes you are right, the program freezes and then resumes. Altough I really had some crashes including data loss. I think one time it did crash directly and the other times I got the Windows-Popup "The program is not responding" after trying to force the program to respond while it was frozen (i.e. wildly clicking on any buttons..). After this it did not unfreeze anymore, even if I kept waiting.

[Simon]

Also another bug which I noticed: When I double-click on a certain point in the timeline to jump to the according time in the video sometimes this opens the context menu. Sometimes it doesn't. It always jumps to the correct time, this is no problem, but I don't get why a double-LEFT-click sometimes in addition opens the context menu.

Please don't get me wrong: I love Advene and it really helps me a lot with my work. Of course I would be happy if it worked more smoothly but I'm happy as it is. I'm trying to help by reporting the bugs I find.

Edit: Ah, I get it. Strictly speaking it's no bug. It happens when you move the mouse while it is being pressed the second time. Then you actually perform two things at the same time: First the double click and then whith the second click of the double-click you create a selection which then opens the context menu according to this selection.
So maybe a suggestion: The feature to directly open a context menu after certain actions (like moving an annotation or like in the above case creating a selection) when the mouse button is released is actually quite neat. The only problem: When you don't want to do any of the things proposed in this context menu, the popping up of the context menu is really annoying because you have to click away the context menu before being able to perform the next task. Maybe in most workflows this is no problem. But when I'm working on a 10minute video with about 600 annotations I am clicking away context menus all the time.
For example the movement of annotations: Why not do it the same way it is done in Windows Exlorer? Moving it with LEFT mouse click means "moving" it (you could allow modification of the command via CTRL, ALT, SHIFT). Moving it with RIGHT mouse click gets you to the context menu. Because in my opinion the context menu should allways be associated with the right mouse button. That's the way it is almost everywhere.

[oaubert]

Thanks for your feedback, it is appreciated.

I agree that the context menu should always be associated to the right button, and in fact it it exactly the case: right-clicking on an annotation will bring its context menu, and right-clicking in the timeline background will also popup a context menu. But the menus that you describe are not context menus for elements, but choice menus for actions: either for the action of selecting an area in the timeline, or for the action of moving an annotation. These are different.

The Control/Shift modifiers are used when drag-and-dropping an annotation: it shortcuts the action menu, and does either an annotation copy (with Control) or move (with Shift).

To alleviate your initial issue (small movement of the mouse when double-clicking), I could implement a size check to ignore small selected areas and only consider them as double-clicks. But do you know that there is another way to select the time, by single-clicking in the snapshots area at the top of the timeline ? It is even simpler, less error-prone and more accessible. And it would (indirectly) solve your issue.