Auto Detect

#4
by iiBLACKii - opened

Thanks for Preview First. I have doubt that instead of passing manually like "Detect X" is there possible to detect from list I am providing and that it detect from list . For instance - list - [UI element,
ad,
ad banner,
advertisement,
advertisement with button,
animation,
area,
article,
background,
background image,
badge,
banner,
banner ad,
bar,
blank,
board,
border,
box,
button,
caption,
card,
checkbox,
container,
content,
content area,
content box,
control bar,
decoration,
decorative,
decorative element,
dialogue box,
display,
divider,
drop-down menu,
dropdown,
dropdown button,
dropdown item,
dropdown menu,
dropdown option,
element,
embedded tweet,
empty,
empty area,
empty space,
emptySpace,
empty_space,
example tag,
extension card,
extension display,
filter toggle,
footer,
form,
handle,
header,
headline,
highlight,
hyperlink,
icon,
icon and button,
icon and text,
icon with text,
icons,
image,
image and button,
image and text,
image button,
image card,
image collection,
image with text,
indicator,
info section,
information,
information bar,
information box,
information display,
information panel,
information section,
informational box,
input,
input section,
item,
item display,
label,
layout,
layout element,
line,
link,
link text,
linked text,
links,
list item,
logo,
menu,
menu icon,
menu item,
menu option,
menu tab,
message,
message box,
navigation,
navigation bar,
navigation item,
navigation tab,
notice,
notification,
notification bar,
option,
other,
overlay,
panel,
placeholder,
playlist card,
popup,
post,
preview,
prompt,
radio button,
radio button label,
rating display,
scrollbar,
search bar,
section,
seekbar,
selection box,
separator,
space,
spacer,
status bar,
subheader,
switch,
symbol,
tab,
tabs,
tag,
task,
task item,
teaser,
text,
text and image,
text block,
text element,
text label,
text link,
text section,
text with dropdown,
text with icon,
text with links,
text_label,
tile,
toggle,
toggle button,
toggle switch,
toolbar,
unknown,
verification,
video,
widget] and it can able to detect from this lists and give bounding box ?

iiBLACKii changed discussion status to closed
iiBLACKii changed discussion status to open

tcules3.png consider this as image

Hello,

UI Element detection is not possible. But you can have a look on AskUI: https://docs.askui.com/docs/general/Tutorials/flutter-android-sample-app

image.png

image.png
I already tried with manually bounding box method but its not performing well as LLM Qwen-2-VL Model . Is there any other way to find solution for UI Element Identification

Sign up or log in to comment