Systems

Links and Demo Videos

Universal Kinect Interface (UKI)

Middleware that enables users to control any application through body motions.

FightingICE (AI Competition Platform)

An open-source platform for developing AI in fighting games

Runner

A 2D platform game for health promotion research

Hand Clenching and Spreading Gaming Interface

Motions detected by image processing via a standard webcam.

Piano Learning with AR virtual character

Augmented Reality with a VR headset to enhance keyboard learning through real-time feedback from a virtual character

Science Birds

Game for procedural content generation research

JUSTIN and Lottery

Audience Participation Games With A Purpose (APGWAP) designed to collect data for artwork images

Procedurally Generated City

A procedurally generated city using the Wave Function Collapse algorithm to fight dimentia.

Research Studies on Minecraft

Early Alzheimer’s detection and Aerial Cinematographer AI

Universal Kinect Interface (UKI)

The Universal Kinect Interface (UKI), or Universal Kinect-type Controller by ICE Lab, is middleware that allows users to control any application using body motions as inputs. It translates detected movements into keyboard or mouse events, which are then sent to the target application.

Main Paper

2017, Pujana Paliyawan and Ruck Thawonmas. UKI: Universal Kinect-type controller by ICE lab. Software: Practice and Experience, 47(10):1343– 1363. doi: 10.1002/spe.2474

Publications
  1. 2018, Pujana Paliyawan, Takahiro Kusano, and Ruck Thawonmas. Motion recommender for preventing injuries during motion gaming. IEEE Access, 7:7977–7988. doi: 10.1109/ACCESS.2018.2889166
  2. 2018, Pujana Paliyawan, Worawat Choensawat, and Ruck Thawonmas. Mossar: motion segmentation by using splitting and remerging strategies. Multimedia Tools and Applications, 77(21):27761–27788. doi: 10.1007/s11042-018-5965-x
  3. 2017, Pujana Paliyawan and Ruck Thawonmas. UKI: Universal Kinect-type controller by ICE lab. Software: Practice and Experience, 47(10):1343– 1363. doi: 10.1002/spe.2474
  4. 2020, Pujana Paliyawan, Kingkarn Sookhanaphibarn, Worawat Choensawat, and Ruck Thawonmas. Towards social facilitation in audience participation games: Fighting game AIs whose strength depends on audience responses. In 2020 IEEE Conference on Games (CoG), pages 686–689. IEEE.
  5. 2017, Pujana Paliyawan, Takahiro Kusano, Yuto Nakagawa, Tomohiro Harada, and Ruck Thawonmas. Adaptive motion gaming AI for health promotion. In 2017 AAAI Spring Symposium Series, Stanford University, California USA, March 27-29, pp. 1-6.
  6. 2016, Pujana Paliyawan and Ruek Thawonmas. Towards ergonomic exergaming. In 2016 IEEE 5th Global Conference on Consumer Electronics, pages 1–2. IEEE.
  7. Pujana Paliyawan, Kingkarn Sookhanaphibarn, Worawat Choensawat, and Ruck Thawonmas. Towards universal Kinect interface for fighting games. In 2015 IEEE 4th Global Conference on Consumer Electronics (GCCE), pages 332–333. IEEE, 2015.
  8. 2015, Pujana Paliyawan, Kingkarn Sookhanaphibarn, Worawat Choensawat, and Ruck Thawonmas. Body motion design and analysis for fighting game interface. In 2015 IEEE Conference on Computational Intelligence and Games (CIG), pages 360–367. IEEE.
  9. 2015, Makoto Ishihara, Taichi Miyazaki, Pujana Paliyawan, Chun Yin Chu, Tomohiro Harada, and Ruck Thawonmas. Investigating Kinect-based fighting game ais that encourage their players to use various skills. In 2015 IEEE 4th Global Conference on Consumer Electronics (GCCE), pages 334–335. IEEE.
  10. 2016, Kingkarn Sookhanaphibarn, Worawat Choensawat, Pujana Paliyawan, and Ruck Thawonmas. Virtual reality system for fire evacuation training in a 3d virtual world. In 2016 IEEE 5th Global Conference on Consumer Electronics, pages 1–2. IEEE.
  11. 2017, Jorge Arturo Morán Bravo, Pujana Paliyawan, Tomohiro Harada, and Ruck Thawonmas. Intelligent assistant for providing instructions and recommending motions during full-body motion gaming. In 2017 IEEE 6th Global Conference on Consumer Electronics (GCCE), pages 1–2. IEEE.
  12. 2018, Yunshi Liu, Pujana Paliyawan, Takahiro Kusano, Tomohiro Harada, and Ruck Thawonmas. A personalized method for calorie consumption assessment. AAAI 2018 Spring Symposium on Beyond Machine Intelligence: Understanding Cognitive Bias and Humanity for Well-Being, Stanford University, California USA, Mar. 26-28, pages 247-252.
  13. 2019. “健康促進のためのプレイヤー適応型モーションゲーミングAI (Player-adaptive Motion Gaming AI for Health Promotion),” Game Programming Workshop 2019 (GPW2019).
  14. 2018. モーションゲームのためのユニバーサルスケルトン構築. 平成30年度情報処理学会関西支部大会講演論文集(電子ファイル), G-113, 大阪，2018年9月30日．
  15. 2017. Calorie Consumption Assessment by using Image-processing-based Method for Exerciser and Motion Gamers. ゲーム学会第16回全国大会論文集, pp. 37-38. 2017年11月3日．
  16. 2017. プレイヤー適応型健康促進のMotion Gaming AI. 情報処理学会全国大会 (IPSJ 2017)．

more…

Archived Site

Office Workers Syndrome Prevention

UKI was built based on a Kinect system designed to prevent “Office Workers Syndrome.” It detects prolonged sitting and unhealthy postures through data mining and mathematical models. When unhealthy postures are detected, the system alerts users via popups, voice messages, or an alert device. It also offers enhanced user interfaces and data visualization to generate helpful summary reports.

Publications
  1. 2014, Pujana Paliyawan, Chakarida Nukoolkit, and Pornchai Mongkolnam. Office workers syndrome monitoring using Kinect. In The 20th Asia- Pacific Conference on Communication (APCC2014), pages 58–63. IEEE.
  2. 2014, Pujana Paliyawan, Chakarida Nukoolkit, and Pornchai Mongkolnam. Prolonged sitting detection for office workers syndrome prevention using Kinect. In 2014 11th International Conference on Electrical Engineering/ Electronics, Computer, Telecommunications and Information Technology (ECTI-CON), pages 1–6. IEEE.

more…

Mossar: Motion segmentation by using splitting and remerging strategies

Mossar is a motion segmentation method that combines velocity-based splitting for efficiency and graph-based remerging for accuracy. It allows users to adjust key frames without altering thresholds or rerunning the process. Mossar improves accuracy while reducing complexity and simplifying threshold setting compared to traditional methods. This method was implemented and tested on UKI.

Main Paper

2018, Pujana Paliyawan, Worawat Choensawat, and Ruck Thawonmas. Mossar: motion segmentation by using splitting and remerging strategies. Multimedia Tools and Applications, 77(21):27761–27788. doi: 10.1007/s11042-018-5965-x

Dancing ICE (Dynamic Rehab Game with PCG)

A motion-based rhythm game designed to be used with UKI for home rehabilitation. Players collect procedurally generated nodes based on pre-recorded moves, which can be adjusted by medical experts. The game customizes levels to ensure the player’s movements align with targeted percentages for different body parts.

Main Paper

2020, Anatole Martin, Jean Farines, Pujana Paliyawan, and Ruck Thawonmas. Dancing ICE: A rhythm game to control the amount of movement through pre-recorded healthy moves. In Motion, Interaction and Games (MIG2020), pages 1–2.

 

Github

Virtual Reality of Fire Evacuation Training

A VR platform for fire evacuation training using UKI to simulate real-life movements like walking and opening doors. Set in a Bangkok University building, the system randomly changes the fire’s starting point to train users on various exit routes. Motion controls are designed for effective training.

Main Paper

2016, Kingkarn Sookhanaphibarn, Worawat Choensawat, Pujana Paliyawan, and Ruck Thawonmas. Virtual reality system for fire evacuation training in a 3d virtual world. In 2016 IEEE 5th Global Conference on Consumer Electronics, pages 1–2. IEEE

FightingICE (Fighting Game AI Competition Platform)

FightingICE is an open-source platform for developing AI in fighting games, offering most of the core features found in typical fighting games. Since 2014, it has been used to host an annual AI competition at the IEEE Conference on Games (formerly Computational Intelligence and Games).

Main Paper

2013, Lu F, Yamamoto K, Nomura LH, Mizuno S, Lee Y, and Thawonmas R. Fighting game artificial intelligence competition platform. In IEEE 2nd Global Conference on Consumer Electronics (GCCE), pages 320–323.

 

Publications
  1. 2018, Pujana Paliyawan, Takahiro Kusano, and Ruck Thawonmas. Motion recommender for preventing injuries during motion gaming. IEEE Access, 7:7977–7988. doi: 10.1109/ACCESS.2018.2889166
  2. 2024, Pujana Paliyawan, Ruck Thawonmas, Kingkarn Sookhanaphibarn, and Worawat Choensawat. Audience participation fighting game: Exploring social facilitation for an enhanced APG experience. In *Heliyon*, volume 10, issue 2, pages 1–14. DOI: 10.1016/j.heliyon.2023.e23967.
  3. 2020, Pujana Paliyawan, Kingkarn Sookhanaphibarn, Worawat Choensawat, and Ruck Thawonmas. Towards social facilitation in audience participation games: Fighting game AIs whose strength depends on audience responses. In 2020 IEEE Conference on Games (CoG), pages 686–689. IEEE.
  4. 2017, Pujana Paliyawan, Takahiro Kusano, Yuto Nakagawa, Tomohiro Harada, and Ruck Thawonmas. Adaptive motion gaming AI for health promotion. In 2017 AAAI Spring Symposium Series, Stanford University, California USA, March 27-29, pp. 1-6.
  5. 2016, Pujana Paliyawan and Ruek Thawonmas. Towards ergonomic exergaming. In 2016 IEEE 5th Global Conference on Consumer Electronics, pages 1–2. IEEE.
  6. Pujana Paliyawan, Kingkarn Sookhanaphibarn, Worawat Choensawat, and Ruck Thawonmas. Towards universal Kinect interface for fighting games. In 2015 IEEE 4th Global Conference on Consumer Electronics (GCCE), pages 332–333. IEEE, 2015.
  7. 2015, Pujana Paliyawan, Kingkarn Sookhanaphibarn, Worawat Choensawat, and Ruck Thawonmas. Body motion design and analysis for fighting game interface. In 2015 IEEE Conference on Computational Intelligence and Games (CIG), pages 360–367. IEEE.
  8. 2020, Pujana Paliyawan, Kingkarn Sookhanaphibarn, Worawat Choensawat, and Ruck Thawonmas. Towards social facilitation in audience participation games: Fighting game AIs whose strength depends on audience responses. In 2020 IEEE Conference on Games (CoG), pages 686–689. IEEE.
  9. 2017, Pujana Paliyawan, Takahiro Kusano, Yuto Nakagawa, Tomohiro Harada, and Ruck Thawonmas. Adaptive motion gaming AI for health promotion. In 2017 AAAI Spring Symposium Series, Stanford University, California USA, March 27-29, pp. 1-6.
  10. 2022, Junjie H. Xu, Hong Huang, Xiaoling Ling, and Pujana Paliyawan. Toward collaborative game commentating utilizing pre-trained generative language models. the 40th IEEE International Conference on Consumer Electronics (ICCE 2022), pages 1– 4. doi: 10.1109/ICCE53296.2022.9730353
  11. 2022, Hong Huang, Junjie H Xu, Xiaoling Ling, and Pujana Paliyawan. Sentence punctuation for collaborative commentary generation in esports live-streaming. the 40th IEEE International Conference on Consumer Electronics (ICCE 2022), pages 1– 2. doi: 10.1109/ICCE53296.2022.9730453
  12. 2021, Junjie H Xu, Zhou Fang, Qihang Chen, Satoru Ohno, and Pujana Paliyawan. Fighting game commentator with pitch and loudness adjustment utilizing highlight cues. In 2021 IEEE 10th Global Conference on Consumer Electronics (GCCE), pages 366–370. IEEE, 2021.
  13. 2021, Junjie H Xu, Yulin Cai, Zhou Fang, and Pujana Paliyawan. Promoting mental well-being for audiences in a live-streaming game by highlight- based bullet comments. In 2021 IEEE 10th Global Conference on Consumer Electronics (GCCE), pages 383–385. IEEE.
  14. 2020, Thanat Jumneanbun, Sunee Sae-Lao, Pujana Paliyawan, Ruck Thawonmas, Kingkarn Sookhanaphibarn, and Worawat Choensawat. Rap-style comment generation to entertain game live streaming. In 2020 IEEE Conference on Games (CoG), pages 706–707. IEEE.
  15. 2020, Junjie Xu, Pujana Paliyawan, Yiming Zhang, Ruck Thawonmas, and Tomohiro Harada. Player dominance adjustment motion gaming AI for health promotion. In Motion, Interaction and Games (MIG2020), pages 1–2.
  16. 2017, Jorge Arturo Morán Bravo, Pujana Paliyawan, Tomohiro Harada, and Ruck Thawonmas. Intelligent assistant for providing instructions and recommending motions during full-body motion gaming. In 2017 IEEE 6th Global Conference on Consumer Electronics (GCCE), pages 1–2. IEEE.
  17. 2017, Takahiro Kusano, Pujana Paliyawan, Tomohiro Harada, and Ruck Thawonmas. Towards adaptive motion gaming AI with player’s behavior modeling for health promotion. In 2017 IEEE 6th Global Conference on Consumer Electronics (GCCE), pages 1–2. IEEE.
  18. 2017, Takahiro Kusano, Pujana Paliyawan, Tomohiro Harada, and Ruck Thawonmas. Analysis of relationship between the player’s behavior change and the effectiveness of a health promotion AI. In 2017 Nicograph International (NicoInt), pages 92–92. IEEE.
  19. 2015, Makoto Ishihara, Taichi Miyazaki, Pujana Paliyawan, Chun Yin Chu, Tomohiro Harada, and Ruck Thawonmas. Investigating Kinect-based fighting game ais that encourage their players to use various skills. In 2015 IEEE 4th Global Conference on Consumer Electronics (GCCE), pages 334–335. IEEE.
  20. 2019. “健康促進のためのプレイヤー適応型モーションゲーミングAI (Player-adaptive Motion Gaming AI for Health Promotion),” Game Programming Workshop 2019 (GPW2019).
  21. 2018. モーションゲームのためのユニバーサルスケルトン構築. 平成30年度情報処理学会関西支部大会講演論文集(電子ファイル), G-113, 大阪，2018年9月30日．
  22. 2017. Calorie Consumption Assessment by using Image-processing-based Method for Exerciser and Motion Gamers. ゲーム学会第16回全国大会論文集, pp. 37-38. 2017年11月3日．
  23. 2017. プレイヤー適応型健康促進のMotion Gaming AI. 情報処理学会全国大会 (IPSJ 2017)．

more…

Official Site

Adaptive Motion Gaming AI for Health Promotion

An AI designed to promote balanced use of body segments during full-body motion gaming. Using the FightingICE platform and the UKI middleware for motion control, the AI analyzes the player’s health and predicts actions that encourage balanced movement. It selects its next move based on a Monte-Carlo Tree Search algorithm, aiming to improve the player’s physical engagement.

Main Paper

2017, Pujana Paliyawan, Takahiro Kusano, Yuto Nakagawa, Tomohiro Harada, and Ruck Thawonmas. Adaptive motion gaming AI for health promotion. In 2017 AAAI Spring Symposium Series, Stanford University, California USA, March 27-29, pp. 1-6.

Publications
  1. 2017, Takahiro Kusano, Pujana Paliyawan, Tomohiro Harada, and Ruck Thawonmas. Towards adaptive motion gaming AI with player’s behavior modeling for health promotion. In 2017 IEEE 6th Global Conference on Consumer Electronics (GCCE), pages 1–2. IEEE.
  2. 2017, Takahiro Kusano, Pujana Paliyawan, Tomohiro Harada, and Ruck Thawonmas. Analysis of relationship between the player’s behavior change and the effectiveness of a health promotion AI. In 2017 Nicograph International (NicoInt), pages 92–92. IEEE.
  3. 2019. “健康促進のためのプレイヤー適応型モーションゲーミングAI (Player-adaptive Motion Gaming AI for Health Promotion),” Game Programming Workshop 2019 (GPW2019).
  4. 2018. モーションゲームのためのユニバーサルスケルトン構築. 平成30年度情報処理学会関西支部大会講演論文集(電子ファイル), G-113, 大阪，2018年9月30日．
  5. 2017. プレイヤー適応型健康促進のMotion Gaming AI. 情報処理学会全国大会 (IPSJ 2017)．

Fighting Game AIs whose Strength Depends on Audience Responses

An AI for audience participation fighting games, designed to mimic social facilitation from human psychology. The AIs adjust difficulty based on audience responses and utilize Monte-Carlo Tree Search. Key mechanisms include dynamic difficulty adjustment, promoting human-like behavior, and integrating social facilitation. One AI generalizes an existing AI by adding a health difference parameter, while the other strictly controls score differences for social facilitation. Results show the first AI produces more human-like behavior, while the second better adjusts strength.

Main Paper

2024, Pujana Paliyawan, Ruck Thawonmas, Kingkarn Sookhanaphibarn, and Worawat Choensawat. Audience participation fighting game: Exploring social facilitation for an enhanced APG experience. In *Heliyon*, volume 10, issue 2, pages 1–14. DOI: 10.1016/j.heliyon.2023.e23967.

  1. 2020, Pujana Paliyawan, Kingkarn Sookhanaphibarn, Worawat Choensawat, and Ruck Thawonmas. Towards social facilitation in audience participation games: Fighting game AIs whose strength depends on audience responses. In 2020 IEEE Conference on Games (CoG), pages 686–689. IEEE.
Publications
  1. 2020, Pujana Paliyawan, Kingkarn Sookhanaphibarn, Worawat Choensawat, and Ruck Thawonmas. Towards social facilitation in audience participation games: Fighting game AIs whose strength depends on audience responses. In 2020 IEEE Conference on Games (CoG), pages 686–689. IEEE.

Player Dominance Adjustment

This paper introduces PDAHP-AI, a fighting game AI designed to promote balanced body movement in motion gaming. The AI uses Monte Carlo tree search and the Player Dominance Adjustment (PDA) concept, which adjusts its behavior based on the player’s actions. When the player performs healthy movements, the AI lets them dominate, encouraging them to repeat these actions. Conversely, when unhealthy movements occur, the AI responds with stronger actions. This dynamic approach promotes balanced use of body segments during gameplay by reinforcing healthier physical activity.

Main Paper

2020, Junjie Xu, Pujana Paliyawan, Ruck Thawonmas, and Tomohiro Harada. Player dominance adjustment: Promoting self-efficacy and experience of game players by adjusting dominant power. In 2019 IEEE 8th Global Conference on Consumer Electronics (GCCE), pages 487–488. IEEE.

Procedural Commentary Generation

Procedural Commentary Generation refers to the automated creation of commentary in live-streaming or gaming environments using predefined rules and algorithms. Rather than relying on human commentators, this system dynamically generates real-time commentary based on in-game events, player actions, or audience feedback. FightingICE has been used for research in this field, focusing on collaborative commentating, sentence punctuation in esports streams, and adjusting pitch and loudness based on game highlights. Studies also explore methods to enhance audience engagement and well-being, including innovative approaches like rap-style commentary.

Publications
  1. 2022, Junjie H. Xu, Hong Huang, Xiaoling Ling, and Pujana Paliyawan. Toward collaborative game commentating utilizing pre-trained generative language models. the 40th IEEE International Conference on Consumer Electronics (ICCE 2022), pages 1– 4. doi: 10.1109/ICCE53296.2022.9730353
  2. 2022, Hong Huang, Junjie H Xu, Xiaoling Ling, and Pujana Paliyawan. Sentence punctuation for collaborative commentary generation in esports live-streaming. the 40th IEEE International Conference on Consumer Electronics (ICCE 2022), pages 1– 2. doi: 10.1109/ICCE53296.2022.9730453
  3. 2021, Junjie H Xu, Zhou Fang, Qihang Chen, Satoru Ohno, and Pujana Paliyawan. Fighting game commentator with pitch and loudness adjustment utilizing highlight cues. In 2021 IEEE 10th Global Conference on Consumer Electronics (GCCE), pages 366–370. IEEE, 2021.
  4. 2021, Junjie H Xu, Yulin Cai, Zhou Fang, and Pujana Paliyawan. Promoting mental well-being for audiences in a live-streaming game by highlight- based bullet comments. In 2021 IEEE 10th Global Conference on Consumer Electronics (GCCE), pages 383–385. IEEE.
  5. 2020, Thanat Jumneanbun, Sunee Sae-Lao, Pujana Paliyawan, Ruck Thawonmas, Kingkarn Sookhanaphibarn, and Worawat Choensawat. Rap-style comment generation to entertain game live streaming. In 2020 IEEE Conference on Games (CoG), pages 706–707. IEEE.
Github (ICEAgent)

Runner: A 2D Platform Game for Physical and Mental Health Promotion

“Runner” is a 2D platform game developed to enhance both physical and mental health through motion-based gameplay. The system procedurally generates content to promote balanced body movement and increase variability, ensuring players engage in diverse physical activity. It also incorporates dynamic difficulty adjustment (DDA), which helps players meet fitness goals, such as burning a specific number of calories. Additionally, “Runner” includes features aimed at improving mental well-being by encouraging positive emotions, such as smiling, to enhance enjoyment and overall mental health during gameplay.

Main Paper

2019, Camille El-Habr, Xavier Garcia, Pujana Paliyawan, and Ruck Thawonmas. Runner: A 2D platform game for physical health promotion. SoftwareX, 10:100329. doi: 10.1016/j.softx.2019.100329

Publications
  1. 2020, Sunee Sae-Lao, Thanat Jumneanbun, Pujana Paliyawan, and Ruck Thawonmas. Encourage players to smile while playing games bring more enjoyment. In 2020 IEEE Conference on Games (CoG), pages 690–693. IEEE.
  2. 2019, Sunee Sae-Lao, Pujana Paliyawan, Ruck Thawonmas, and Tomohiro Harada. Enhance physical and mental well-being of game players in an endless running game. In 2019 IEEE 8th Global Conference on Consumer Electronics (GCCE), pages 945–946. IEEE.
  3. 2020, Yiming Zhang, Albertus Agung, Pujana Paliyawan, and Ruck Thawonmas. An audience participation game with difficulty adjustment and rap-style commentary based on audience inputs. In 2021 IEEE 10th Global Conference on Consumer Electronics (GCCE), pages 845–846. IEEE.

more…

Github (Original Runner)
Game (Smile Runner)

Play with Smile

We introduces a smile-based interface to enhance player experience in the endless runner game, “Runner.” The smile mechanism allows players to trigger a boost by smiling, using the Affdex deep-learning facial recognition toolkit. The game was tested in three modes: Standard (no smile or boost), Smile (boost triggered by smiling), and Auto (automatic boosting). Results indicate that the smile mechanism significantly improves gameplay enjoyment.

Publications
  1. 2020, Sunee Sae-Lao, Thanat Jumneanbun, Pujana Paliyawan, and Ruck Thawonmas. Encourage players to smile while playing games bring more enjoyment. In 2020 IEEE Conference on Games (CoG), pages 690–693. IEEE.
  2. 2019, Sunee Sae-Lao, Pujana Paliyawan, Ruck Thawonmas, and Tomohiro Harada. Enhance physical and mental well-being of game players in an endless running game. In 2019 IEEE 8th Global Conference on Consumer Electronics (GCCE), pages 945–946. IEEE.

Gaming Interface Utilizing Hand Clenching and Spreading

This paper explores how playing games using hand motions can enhance brain memory. Building on psychological research suggesting that hand exercises positively impact brain function and memory, the authors developed an interface that detects hand movements through image processing, translating them into keyboard commands for any existing game, requiring only a standard webcam. An experiment was conducted using a jumping game to compare three interfaces: a traditional keyboard, a Kinect for hand-raising motions, and the new interface for hand clenching and spreading motions.

Main Paper

2020, Yunshi Liu, Febri Abdullah, Pujana Paliyawan, Ruck Thawonmas, and Tomohiro Harada. Improving brain memory through gaming using hand clenching and spreading. In Motion, Interaction and Games (MIG2020), pages 1–2.

Github

Piano learning application with feedback provided by an AR virtual character

A mobile application that uses Augmented Reality to enhance keyboard learning. Utilizing VR headset, the app visualizes a virtual character that gives real-time feedback based on the user’s performance. Users can choose from pre-defined songs and learn through a simplified notation, making the process fun and engaging while promoting motivation in music education.

Main Paper

2016, Carlos A Torres Fernandez, Pujana Paliyawan, Chu Chun Yin, and Ruck Thawonmas. Piano learning application with feedback provided by an AR virtual character. In 2016 IEEE 5th Global Conference on Consumer Electronics, pages 1–2. IEEE.

Science Birds

This system, mainly used for procedural content generation research, is based on the original *Angry Birds* game. It features various game objects like birds with special abilities, pigs with different health points, and blocks made from materials with varying physical properties. TNTs cause area damage. These objects are placed on different terrains, from flat to complex. *Science Birds* supports only some of these objects for research purposes.

Publications
  1. 2020, Febri Abdullah, Pujana Paliyawan, Ruck Thawonmas, and Fitra A Bachtiar. Generating Angry Birds-like levels with domino effects using constrained novelty search. In 2020 IEEE Conference on Games (CoG), pages 698–701. IEEE.
  2. 2020, Nowshin Faiza Alam, Albertus Agung, Febri Abdullah, Pujana Paliyawan, and Ruck Thawonmas. Singing with an Angry-birds-like game. In 2020 IEEE Conference on Games (CoG), pages 772–773. IEEE.
  3. 2020, Zhou Fang, Pujana Paliyawan, Ruck Thawonmas, and Tomohiro Harada. Towards an Angry-birds-like game system for promoting mental wellbeing of players using art-therapy-embedded procedural content generation. In 2019 IEEE 8th Global Conference on Consumer Electronics (GCCE), pages 947–948. IEEE.
  4. 2019, Febri Abdullah, Pujana Paliyawan, Ruck Thawonmas, Tomohiro Harada, and Fitra A Bachtiar. An Angry Birds level generator with Rube Goldberg machine mechanisms. In 2019 IEEE Conference on Games (CoG), pages 1–8. IEEE.
  5. 2019, Jingdi Xu, Yuuki Okido, Sunee Sae-Lao, Pujana Paliyawan, Ruck Thawonmas, and Tomohiro Harada. Promoting emotional well-being with Angry-birds-like gameplay on pixel image levels. In 2019 IEEE 7th International Conference on Serious Games and Applications for Health (SeGAH), pages 1–4. IEEE.
  6. 2019, Febri Abdullah, Changeun Yang, Pujana Paliyawan, Ruck Thawonmas, Tomohiro Harada, and Fitra A Bachtiar. Effect of Angry Birds-like live streaming on working memory. In 2019 SEGAH, pp.1–2.
  7. 2019, Febri Abdullah, Changeun Yang, Pujana Paliyawan, Ruck Thawonmas, Tomohiro Harada, and Fitra A Bachtiar. Promoting emotions with Angry Birds-like gameplay on Rube Goldberg machine levels. In 2019 IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia), pages 149–150. IEEE.
  8. 2019, Changeun Yang, Pujana Paliyawan, Ruck Thawonmas, and Tomohiro Harada. TGIF!: Selecting the most healing TNT by optical flow. In AAAI Spring Symposium: Interpretable AI for Well-being.
  9. 2018, Changeun Yang, Pujana Paliyawan, Tomohiro Harada, and Ruck Thawonmas. Blow up depression with in-game TNTs. In 2018 IEEE 7th Global Conference on Consumer Electronics (GCCE), pages 820–821. IEEE.
  10. 2018, Jingdi Xu, Changeun Yang, Yuuki Okido, Pujana Paliyawan, Ruck Thawonmas, and Tomohiro Harada. An Angry Birds-like game system for promoting players’ emotion. In 2018 IEEE 7th Global Conference on Consumer Electronics (GCCE), pages 159–160. IEEE, 2018.
  11. 2018, Yuxuan Jiang, Pujana Paliyawan, Tomohiro Harada, and Ruck Thawonmas. An audience participation Angry Birds platform for social well-being. In the 19th annual European GAME-ON Conference on Simulation and AI in Computer Games (GAMEON’2018). September, pages 18–20.
  12. 2018, Changeun Yang, Yuxuan Jiang, Pujana Paliyawan, Tomohiro Harada, and Ruck Thawonmas. Smile with Angry Birds: Two smile-interface implementations. In 2018 Nicograph International (NicoInt), pages 80– 80. IEEE.
Github

Generating Levels With Domino Effects

A method to generate diverse, Rube Goldberg-inspired levels for an Angry Birds-like game using Constrained Novelty Search (CNS). These levels focus on a domino effect, allowing completion with a single bird shot. By evolving both feasible and infeasible populations, the method produces levels with greater block-type diversity while maintaining stability and a perfect-shot rate. Results show that this approach generates more diverse and playable levels compared to previous methods.

Main Paper

2019, Febri Abdullah, Pujana Paliyawan, Ruck Thawonmas, Tomohiro Harada, and Fitra A Bachtiar. An Angry Birds level generator with Rube Goldberg machine mechanisms. In 2019 IEEE Conference on Games (CoG), pages 1–8. IEEE.

Publications
  1. 2020, Febri Abdullah, Pujana Paliyawan, Ruck Thawonmas, and Fitra A Bachtiar. Generating Angry Birds-like levels with domino effects using constrained novelty search. In 2020 IEEE Conference on Games (CoG), pages 698–701. IEEE.
  2. 2019, Febri Abdullah, Pujana Paliyawan, Ruck Thawonmas, Tomohiro Harada, and Fitra A Bachtiar. An Angry Birds level generator with Rube Goldberg machine mechanisms. In 2019 IEEE Conference on Games (CoG), pages 1–8. IEEE.
  3. 2019, Febri Abdullah, Changeun Yang, Pujana Paliyawan, Ruck Thawonmas, Tomohiro Harada, and Fitra A Bachtiar. Effect of Angry Birds-like live streaming on working memory. In 2019 SEGAH, pp.1–2.
  4. 2019, Febri Abdullah, Changeun Yang, Pujana Paliyawan, Ruck Thawonmas, Tomohiro Harada, and Fitra A Bachtiar. Promoting emotions with Angry Birds-like gameplay on Rube Goldberg machine levels. In 2019 IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia), pages 149–150. IEEE.

Art-Therapy-embedded PCG

Integrates art therapy with a video game to promote mental well-being. Players draw images that are converted into levels for an Angry Birds-like game. The system has three components: a drawing recognizer (Sketcher) that identifies objects, a level generator (PCG AI) that turns drawings into game levels, and an Art Therapy AI that provides positive feedback to enhance players’ emotions. The system aims to combine creativity with gaming to support mental health.

Main Paper

2020, Zhou Fang, Pujana Paliyawan, Ruck Thawonmas, and Tomohiro Harada. Towards an Angry-birds-like game system for promoting mental wellbeing of players using art-therapy-embedded procedural content generation. In 2019 IEEE 8th Global Conference on Consumer Electronics (GCCE), pages 947–948. IEEE.

Github

Level Generation from Sketches with CycleGAN

This paper enhances a Sketch-to-Level Generator for an Angry Birds-like game by using Cycle-Consistent Adversarial Networks (CycleGAN). CycleGAN is trained on two datasets: sketch drawings from Google’s Quick, Draw! and typical level structures from AIBIRDS level generation competitions. The output from the trained CycleGAN is used as input for the Sketch-to-Level Generator, enabling it to generate more successful game levels from arbitrary sketches

IEEE GCCE 2020 Excellent Student Poster Award, Gold Prize

Main Paper

2020, Mury F Dewantoro, Febri Abdullah, Pujana Paliyawan, Ruck Thawonmas, and Fitra A Bachtiar. Enhancement of Angry Birds level generation from sketches using cycle-consistent adversarial networks. In 2020 IEEE 9th Global Conference on Consumer Electronics (GCCE), pages 564– 565. IEEE.

Singing with an Angry-Birds-like Game

Uses singing as an input to control bird shots in an Angry Birds-like game. Singing, known to reduce stress, is analyzed and used to adjust the shooting performance, promoting mental health and rehabilitation. The system aims to reduce anxiety and boost positive emotions through gameplay. Future work includes testing dynamic difficulty adjustment and evaluating the mental health benefits for players.

Main Paper

2020, Nowshin Faiza Alam, Albertus Agung, Febri Abdullah, Pujana Paliyawan, and Ruck Thawonmas. Singing with an Angry-birds-like game. In 2020 IEEE Conference on Games (CoG), pages 772–773. IEEE.

Smile with Angry Birds

These studies focus on enhancing emotional well-being through gameplay in Angry Birds-like games. They explore innovative mechanics such as using smiles as input for gameplay, procedural content generation to create engaging levels, and algorithms for selecting entertaining TNT explosions. User studies show that these approaches can significantly improve positive emotions while reducing negative feelings, highlighting the potential for integrating emotional factors into game design to promote mental health.

Publications
  1. 2018, Yuxuan Jiang, Pujana Paliyawan, Tomohiro Harada, and Ruck Thawonmas. An audience participation Angry Birds platform for social well-being. In the 19th annual European GAME-ON Conference on Simulation and AI in Computer Games (GAMEON’2018). September, pages 18–20.

An Audience Participation Angry Birds

This paper introduces an audience participation platform for Angry Birds to enhance social well-being. It simplifies gameplay by allowing audiences to control bird shots and create original levels via chat, addressing issues of complexity and fixed content in existing games. The findings confirm that these mechanisms effectively promote cooperation and engagement among players.

Publications
  1. 2019, Jingdi Xu, Yuuki Okido, Sunee Sae-Lao, Pujana Paliyawan, Ruck Thawonmas, and Tomohiro Harada. Promoting emotional well-being with Angry-birds-like gameplay on pixel image levels. In 2019 IEEE 7th International Conference on Serious Games and Applications for Health (SeGAH), pages 1–4. IEEE.
  2. 2019, Changeun Yang, Pujana Paliyawan, Ruck Thawonmas, and Tomohiro Harada. TGIF!: Selecting the most healing TNT by optical flow. In AAAI Spring Symposium: Interpretable AI for Well-being.
  3. 2018, Changeun Yang, Pujana Paliyawan, Tomohiro Harada, and Ruck Thawonmas. Blow up depression with in-game TNTs. In 2018 IEEE 7th Global Conference on Consumer Electronics (GCCE), pages 820–821. IEEE.
  4. 2018, Jingdi Xu, Changeun Yang, Yuuki Okido, Pujana Paliyawan, Ruck Thawonmas, and Tomohiro Harada. An Angry Birds-like game system for promoting players’ emotion. In 2018 IEEE 7th Global Conference on Consumer Electronics (GCCE), pages 159–160. IEEE, 2018.
  5. 2018, Changeun Yang, Yuxuan Jiang, Pujana Paliyawan, Tomohiro Harada, and Ruck Thawonmas. Smile with Angry Birds: Two smile-interface implementations. In 2018 Nicograph International (NicoInt), pages 80– 80. IEEE.

JUSTIN (APGWAP)

This JUSTIN is (Japanese Ukiyo-e Streaming That Improves Narrative) a game system designed to collect descriptive data for artwork images, specifically Japanese ukiyo-e, through audience participation on a live streaming platform. It combines the concepts of Audience Participation Games and Games With A Purpose (APGWAP) to solve problems that computers struggle with, utilizing human input. The system helps gather valuable data that can support research in the humanities and promote cultural heritage, addressing the current lack of quality descriptions for ukiyo-e images.

Main Paper

2020, Ngoc Cuong Nguyen, Ruck Thawonmas, Pujana Paliyawan, and Hai V Pham. JUSTIN: An audience participation game with a purpose for collecting descriptions for artwork images. In 2020 IEEE Conference on Games (CoG), pages 344–350. IEEE.

Publications
  1. 2022, Albertus Agung, Roman Savchyn, Pujana Paliyawan, and Ruck Thawonmas. Cute Helper: A Study on the Effect of Virtual Character Expressions on Players’ Engagement in a Game for Collecting Artwork Descriptions. Proc. of the 2022 IEEE Conference on Games (CoG 2022), online, pp. 588-589, Aug. 21-24, 2022. doi: 10.1109/CoG51982.2022.9893683.
  2. 2021, Albertus Agung, Thanat Jumneanbun, Pujana Paliyawan, and Ruck Thawonmas. JUSTIN sings: adding a musical-style narration generator to an APGWAP that collects ukiyo-e artworks descriptions. In 2021 IEEE 10th Global Conference on Consumer Electronics (GCCE), pages 421– 422. IEEE.
  3. 2020, Roman Savchyn, Pujana Paliyawan, and Ruck Thawonmas. Cuteness in JUSTIN: Adding a virtual moderator for entertaining user in an audience participation game for collecting artwork description data. In 2021 IEEE 10th Global Conference on Consumer Electronics (GCCE), pages 843– 844. IEEE.
  4. 2019, Ngoc Cuong Nguyen, Pujana Paliyawan, Ruck Thawonmas, Hai V Pham, Harada Tomohiro, Keiko Suzuki, and Masaaki Kidachi. Potentials of games with a purpose and audience participation games for descriptive data collection in humanities research. In Japanese Association for Digital Humanities Conference, pages 104–106.
  5. 2019, Ngoc Cuong Nguyen, Zhenao Wei, Pujana Paliyawan, Hai V Pham, Ruck Thawonmas, and Tomohiro Harada. Using GWAP to generate informative descriptions for artwork images on a live streaming platform. In 2019 IEEE International Conference on Consumer Electronics-Asia (ICCEAsia), pages 43–44. IEEE.

more…

Github

Lottery (APGWAP) A Card Game for Collecting Human-Perceived Similarity Data of Artwork Images

This JUSTIN is (Japanese Ukiyo-e Streaming That Improves Narrative) a game system designed to collect descriptive data for artwork images, specifically Japanese ukiyo-e, through audience participation on a live streaming platform. It combines the concepts of Audience Participation Games and Games With A Purpose (APGWAP) to solve problems that computers struggle with, utilizing human input. The system helps gather valuable data that can support research in the humanities and promote cultural heritage, addressing the current lack of quality descriptions for ukiyo-e images.

Main Paper

2022, Zhenao Wei, Pujana Paliyawan, Ruck Thawonmas. A Card Game for Collecting Human-Perceived Similarity Data of Artwork Images. IEEE Access, vol. 10, pp. 8103-8111. doi: 10.1109/ACCESS.2021.3055813.

Publications
  1. 2022, Zhenao Wei, Pujana Paliyawan, Ruck Thawonmas. Improving Deep-Feature Image Similarity Calculation: A Case Study on an Ukiyo-e Card Matching Game Lottery. IEEE Access, vol. 10, pp. 44608-44616. doi: 10.1109/ACCESS.2022.3131822.
  2. 2022, Zhenao Wei, Shizhe Wang, Pujana Paliyawan, and Ruck Thawonmas. Difference in Perceived Similarity Between Humans and Machines―A Case Study on Ukiyo-e Artworks, ART RESEARCH, vol. 22-2, 8 pages. doi: 10.24432/issn.2189-228X.2022.22-2.41.
  3. 2019, Zhenao Wei, Yuntian Ma, Shizhe Wang, Nogc Cuong Nguyen, Pujana Paliyawan, Ruck Thawonmas, Tomohiro Harada, Keiko Suzuki, and Masaaki Kidachi. Establishment of an ukiyo-e similarity database by a game with a purpose. In Proc. of Japanese Association for Digital Humanities Conference, volume 2019, pages 55–58.

more…

Infinite Procedurally Generated City

An infinite, procedurally generated city, assembled out of blocks using the Wave Function Collapse algorithm with backtracking.

Citation

Kleineberg M. Infinite procedurally generated city with the wave function collapse algorithm.
Internet: https://marian42. de/article/wfc/[May. 29, 2020]. 2019.

Site
Github

PCG-driven 3D game for preclinical detection of dementia

This paper introduces a 3D game designed for cognitive assessment to detect cognitive decline, particularly in aging individuals. The game integrates neuropsychological screening tests to evaluate cognitive processing, executive function, and visual attention. Utilizing procedural content generation, it offers a unique experience each time, promoting player engagement and continuous play. This approach aims to enhance early detection of cognitive decline, ultimately aiding in effective treatment.

IEEE GCCE 2021 Excellent Demo! Award, Gold Prize

Main Paper

2020, Tannop Sangvanloy, Pujana Paliyawan, Ruck Thawonmas, Kingkarn Sookhanaphibarn, and Worawat Choensawat. Toward a PCG-driven 3D game for preclinical detection of dementia. In 2021 IEEE 10th Global Conference on Consumer Electronics (GCCE), pages 711–712. IEEE.

A maze game with singing interface to fight dementia

This paper presents a puzzle game for early dementia detection, featuring a singing interface to enhance mental health. It emphasizes spatial navigation as crucial for early detection, with gameplay logs used to assess cognitive states. A mini-game offers hints when players get lost, reducing stress during cognitive assessment. The game uses the Wave Function Collapse algorithm to generate mazes, incorporating a Japanese architectural style and Ukiyo-e artworks. Players must find a floating Ukiyo-e artwork at the maze exit, promoting both an engaging experience and Japanese cultural heritage.

Main Paper

2020, Jaturawit Chaiwong, Pujana Paliyawan, Ruck Thawonmas, Kingkarn Sookhanaphibarn, and Worawat Choensawat. A maze game with singing interface to fight dementia. In 2021 IEEE 10th Global Conference on Consumer Electronics (GCCE), pages 847–848. IEEE.

Research Studies on Minecraft

We conducted research on Minecraft, a widely recognized sandbox game known for its creativity and exploration, focusing on developing a mod for early Alzheimer’s detection through spatial navigation. Additionally, we created an Aerial Cinematographer AI for automated filming of settlements in the Generative Design in Minecraft Competition, enhancing the evaluation process for complex designs.

 

Toward a Minecraft mod for early detection of Alzheimer’s disease in young adults

This paper presents a Minecraft-based system for early detection of Alzheimer’s disease in young adults, emphasizing the critical role of spatial navigation in prevention. The proposed system is compared to the established Sea Hero Quest (SHQ) by analyzing players’ navigational patterns, revealing a strong correlation between the two, suggesting the proposed system’s effectiveness.

Players enter a game world with a map and signboard showing starting points and checkpoints, guiding them on item collection. Approaching checkpoints, they find marked areas with chests and signboards displaying checkpoint IDs. Opening a chest reveals its contents, allowing players to select the correct item for their inventory. After collecting items, players use a crafting table at the final checkpoint to create a new item, dependent on correct item acquisition; otherwise, a different item is produced. The game concludes once the crafted item is added to the inventory.

Main Paper

2022, Satoko Ito, Ruck Thawonmas, and Pujana Paliyawan. Toward a Minecraft mod for early detection of Alzheimer’s disease in young adults. arXiv preprint arXiv:2201.10148, 2022 (MIG2021 Poster), pages 1– 2.

Site
Github

An Aerial Cinematographer AI for Settlements

This paper presents a system for automated aerial shooting of Minecraft settlements, particularly those created for the Generative Design in Minecraft Competition (GDMC). The system uses a decision tree method to learn the movement patterns of players or GDMC judges during settlement assessments. The paper introduces an aerial cinematographer AI trained through supervised learning, showing effective performance in capturing videos of settlements, especially when positioned near the settlement center. Additional details are provided in a supplementary page, which contains information not included in the two-page demo paper submitted for review.

Main Paper

2020, Hao Jia, Ruck Thawonmas, and Pujana Paliyawan. An aerial cinematographer AI for settlements in minecraft–toward their crowd assessment. In 2021 IEEE 10th Global Conference on Consumer Electronics (GCCE), pages 853–854. IEEE.

Site
Github