News!

We have released v1.0.0 version of UFO!

• We can our documentation ready!
• We now support web api to craft content from web page!

What's Changed

• Vyokky/dev by @vyokky in #96
• Bug fixed & requirement updated by @vyokky in #97
• Fix typo in api README.md by @TigeR0se in #98
• merge by @vyokky in #99
• Documentation and fix prompt by @vyokky in #101
• documentation by @vyokky in #103
• Fix prompt and create documentation by @vyokky in #102
• Vyokky/dev by @vyokky in #105
• Documentation updated by @vyokky in #106
• Vyokky/dev by @vyokky in #107
• Pre release by @vyokky in #108
• Vyokky/dev by @vyokky in #109
• New Release for v1.0.0 by @vyokky in #110

New Contributors

• @TigeR0se made their first contribution in #98

Full Changelog: v0.2.1...v1.0.0

New Release for v0.2.1! We are excited to announce the release of version 0.2.1! This update includes several new features and improvements:

1. **HostAgent Refactor:** We've refactored the HostAgent to enhance its efficiency in managing AppAgents within UFO.
2. **Evaluation Agent:** Introducing an evaluation agent that assesses task completion and provides real-time feedback.
3. **Google Gemini Support:** UFO now supports Google Gemini as the inference engine. Refer to our detailed guide in [README.md](/model_worker/readme.md).
4. **Customized User Agents:** Users can now create customized agents by simply answering a few questions.

What's Changed

• docs: update README.md by @eltociear in #64
• Vyokky/dev by @vyokky in #71
• bug fix by @vyokky in #72
• add evaluation by @ShilinHe in #74
• fix qwen bug by @Mac0q in #75
• Update README.md by @vyokky in #78
• Evaluation agent / xml saving / Blackboard by @vyokky in #77
• Bump requests from 2.31.0 to 2.32.0 by @dependabot in #76
• typo by @Mac0q in #79
• Vyokky/fixbug by @vyokky in #80
• Refactor agent state and workflow by @vyokky in #81
• Vyokky/dev Bug fixed by @vyokky in #82
• Bug fixed by @vyokky in #83
• Customization and Asking questions by @vyokky in #88
• gemini support by @Mac0q in #90
• Vyokky/dev Improve HostAgent and pre-release by @vyokky in #92
• Vyokky/dev by @vyokky in #94
• rm procrssor by @vyokky in #95
• Release 0.2.1 by @vyokky in #93

New Contributors

• @ShilinHe made their first contribution in #74

Full Changelog: v0.1.1...v0.2.1

Highlights

New Release for v0.1.1! We've made some significant updates! Previously known as AppAgent and ActAgent, we've rebranded them to HostAgent and AppAgent to better align with their functionalities. Explore the latest enhancements:
1. Learning from Human Demonstration: UFO now supports learning from human demonstration! Utilize the Windows Step Recorder to record your steps and demonstrate them for UFO. Refer to our detailed guide in README.md for more information.
2. Win32 Support: We've incorporated support for Win32 as a control backend, enhancing our UI automation capabilities.
3. Extended Application Interaction: UFO now goes beyond UI controls, allowing interaction with your application through keyboard inputs and native APIs! Presently, we support Word (examples), with more to come soon. Customize and build your own interactions.
4. Control Filtering: Streamline LLM's action process by using control filters to remove irrelevant control items. Enable them in config_dev.yaml under the control filtering section at the bottom.

What's Changed

• et_text enabled by @Lenny2Liu in #35
• price by @Mac0q in #42
• Enable human demonstration RAG by @yunhao0204 in #49
• Bump pillow from 10.2.0 to 10.3.0 by @dependabot in #52
• Vyokky/dev Agent and automator modularization + Learning for demonstration by @vyokky in #50
• update function name and add demo video in README by @yunhao0204 in #54
• hosted api by @Mac0q in #55
• enable win32 backend support by @yunhao0204 in #59
• Update README.md by @vyokky in #60
• control filter by @Mac0q in #61
• New openfileapp by @Lenny2Liu in #57
• fix filter by @Mac0q in #62
• Code refactor and control filtered by @vyokky in #63
• Minor fix for releasing new version by @vyokky in #67
• Dev/reformat by @vyokky in #69
• New release for v0.2.0 by @vyokky in #70

New Contributors

• @dependabot made their first contribution in #52

Full Changelog: v0.0.1...v0.2.0

What's Changed

• et_text enabled by @Lenny2Liu in #35
• price by @Mac0q in #42
• Enable human demonstration RAG by @yunhao0204 in #49
• Bump pillow from 10.2.0 to 10.3.0 by @dependabot in #52
• Vyokky/dev Agent and automator modularization + Learning for demonstration by @vyokky in #50
• update function name and add demo video in README by @yunhao0204 in #54
• hosted api by @Mac0q in #55
• enable win32 backend support by @yunhao0204 in #59
• Update README.md by @vyokky in #60
• control filter by @Mac0q in #61
• New openfileapp by @Lenny2Liu in #57
• fix filter by @Mac0q in #62
• Code refactor and control filtered by @vyokky in #63
• Minor fix for releasing new version by @vyokky in #67
• Dev/reformat by @vyokky in #69
• New release for v0.2.0 by @vyokky in #70

New Contributors

• @dependabot made their first contribution in #52

Full Changelog: v0.0.1...v0.1.1

What's Changed

Key Updates:

1. We now support creating your help documents for each Windows application to become an app expert. Check the [README](./learner/README.md) for more details!
2. UFO now supports RAG from offline documents and online Bing search.
3. You can save the task completion trajectory into its memory for UFO's reference, improving its future success rate!
4. You can customize different GPT models for AppAgent and ActAgent. Text-only models (e.g., GPT-4) are now supported!

Detailed contributions

• Update README.md by @eltociear in #1
• fix: word spelling in prompt by @saifeiLee in #6
• Fix colorama init by @lserinol in #10
• Modify the configuration of AOAI API base and modify the dict of app_selection prompt by @kangyu in #12
• Add config file to ignore, and added a template config file for AAD login method by @Mac0q in #16
• Develop the RAG with offline doc and online bing search by @vyokky in #14
• Develop reformating prompts and readme for offline learner by @vyokky in #19
• readme by @vyokky in #23
• Vyokky/dev: Support Non-visual mode by @vyokky in #24
• OpenAI API by @Mac0q in #25
• check in openfile.py by @Lenny2Liu in #26
• Vyokky/dev Experience Learning by @vyokky in #28
• Add record processor to parse the user behavior record to JSON file by @yunhao0204 in #31
• openai config by @Mac0q in #32
• Fix the prompts by @AL-377 in #33
• Vyokky/executor by @vyokky in #34
• Vyokky/dev v0.0.1 New release by @vyokky in #36
• release v0.0.1 by @vyokky in #38

New Contributors

• @eltociear made their first contribution in #1
• @saifeiLee made their first contribution in #6
• @lserinol made their first contribution in #10
• @kangyu made their first contribution in #12
• @Mac0q made their first contribution in #16
• @vyokky made their first contribution in #14
• @Lenny2Liu made their first contribution in #26
• @AL-377 made their first contribution in #33

Full Changelog: https://github.com/microsoft/UFO/commits/v0.0.1