Projects

Codejam 14 Hackathon Winner🥳🎉

Instructions: This project was part of the Matador AI LLM Sponsored Challenge during Codejam 14, where the goal was to create an AI-powered car finder tool to help customers find their ideal vehicles from a dealership’s inventory. The solution aimed to deliver results through an engaging conversational interface powered by Natural Language Processing (NLP). By integrating a dynamic, multi-step pipeline, the tool interpreted customer needs, updated preferences, and generated highly personalized recommendations in real-time. My team and I won the Matador AI LLM Sponsored Challenge!

Devpost
Frontend Repository
Backend Repository

Front End

Purpose: The frontend served as the primary user interface, enabling users to interact with the AI-powered car finder tool seamlessly.

Implementation:

  • Framework: Built with Next.js, the frontend provided a fast and modern UI optimized for responsiveness and usability.
  • User Interaction: Users entered their preferences (e.g., “Show me SUVs under $30,000 with leather seats”) via text or buttons.
  • API Calls: The frontend forwarded user inputs to the backend through API endpoints, initiating the NLP pipeline for processing.

Why:

  • Next.js allowed for a robust and intuitive interface, ensuring users could interact naturally with the AI tool while benefiting from fast responses and a smooth experience.

Custom Named Entity Recognition (NER)

Purpose: Extract meaningful entities (e.g., car features, budget, type) from user inputs, enabling the tool to understand specific customer requirements.

Implementation:

  • Custom NLP Model: A custom NER model was trained in Python using manually annotated datasets tailored to vehicle specifications (e.g., “SUV,” “low mileage,” “electric car”).
  • Task: Tokenize key phrases and identify entities that represented vehicle features or constraints (e.g., “red convertible” → {color: red, type: convertible}).
  • Integration: The identified entities were passed to the Sentiment Analysis step for further evaluation.

Why:

  • A custom NER ensured high accuracy in recognizing terms specific to the automotive domain.

Sentiment Analysis (SA)

Purpose: Determine whether users expressed a positive or negative sentiment toward each identified token (e.g., “I don’t want leather seats”).

Implementation:

  • Sentiment Analysis Model: An NLP-based SA model evaluated the sentiment of each tokenized feature extracted by the NER.
  • Positive/Negative Categorization: Tokens were classified into positive (desired features) or negative (undesired features) categories.
  • Output: Each token’s sentiment was used to inform the encoder stage.

Why:

  • By understanding user sentiment, the tool ensured it only prioritized features the customer wanted, avoiding irrelevant or unwanted suggestions.

Encoder

Purpose: Transform each token (including its sentiment) into a unique vector representation within a multidimensional vector space.

Implementation:

  • Vector Space Encoding: Tokens were encoded into vectors based on their sentiment and context. Positive features contributed positively to the preference vector, while negative features had the opposite effect.
  • Semantic Context: The encoder ensured that user preferences were mathematically represented, allowing precise matching with dealership inventory.

Why:

  • Encoding user preferences into vectors allowed the system to quantify qualitative inputs, making them comparable with vehicle attributes.

Customer Preference Vector

Purpose: Maintain and dynamically update a vector representation of customer preferences based on both previously expressed needs and new inputs.

Implementation:

  • Matrix Multiplication: The updated vectors from the encoder were multiplied with the existing preference vector, dynamically integrating new requirements.
  • Contextual Memory: By retaining a history of user interactions, the preference vector ensured that all user inputs were accounted for, even as new prompts were received.

Why:

  • Continuously updating the preference vector ensured the system adapted to evolving user needs while retaining relevant historical context.

Final Output Back to the User

Purpose: Generate and deliver a conversational response with relevant car recommendations based on the updated preference vector.

Implementation:

  • GPT Wrapper: The final response was generated by an AI language model, which processed the updated preference vector to create a natural, conversational reply.
  • Content: The output included vehicle suggestions, comparisons, or follow-up questions (e.g., “The Toyota RAV4 might be perfect for you. Would you like to know more?”).
  • Frontend Integration: The formatted response was returned to the Next.js frontend for display.

Why:

  • Using a conversational AI for responses ensured a user-friendly experience that felt interactive and engaging, mimicking the expertise of a human car salesperson.

Robo Rover 🤖

This robot was designed for the ECSE 211 course to autonomously navigate a specified course, where it was required to collect ping pong balls at a dispensing zone and subsequently launch them into baskets at a designated launching zone. The course included tunnels, and the robot needed to independently identify and traverse the correct tunnel to reach both zones effectively. The robot’s components were controlled by using a Raspberry Pi. Key constraints included:

  • Size Limitations: The robot’s height could not exceed 25 cm, and its width was restricted to less than 21 cm to ensure it could navigate through the tunnel.
  • Material Constraints: The build was limited to components from a DPM kit, along with cardboard, masking tape, and elastics.
  • Operational Constraints: The robot was required to complete its task within a maximum duration of 3 minutes and needed to have the capacity to carry and launch up to 10 ping pong balls.
  • Sensor and Motor Constraints: The design could incorporate no more than 4 sensors and 4 motors.

 

Launching System

Purpose: The launch system was specifically designed to load and launch ping pong balls into targets. It needed to store balls, align them for launching, and then project them with enough force to reach the target efficiently. The first bounce of each ball counted towards the final score.

Motors:

    • 1 Medium Motor: This motor was responsible for aligning each ping pong ball into the launching mechanism, ensuring they were correctly positioned for launch.
    • 1 Large Motor: Operated the launching mechanism, which propelled the balls towards their target.

Software Overview

The robot’s operations were governed by software written in Python, utilizing specific algorithms for navigation, including:

  • Wall following algorithms to navigate along the course using ultrasonic sensors.
  • Gyroscopic feedback for accurate orientation and maneuvering.
  • Color detection algorithms to identify the correct launching areas via color sensors.

This software setup allowed the robot to perform its tasks fully autonomously, ideally responding to environmental cues without human intervention, and aligning precisely for effective ball launching.

Navigation System

Purpose: The navigation system was crucial for the robot’s autonomous movement through the course, involving tunnel detection, obstacle avoidance, and precise positioning for ball dispensing and launching.

Motors:

  • 2 Large Motors: These motors provided the necessary movement capabilities, allowing the robot to traverse the course as planned.

Sensors:

  • 2 Ultrasonic Sensors: Essential for detecting obstacles and navigating through tunnels by measuring distances to walls.
  • 1 Gyro Sensor: Used to maintain the robot’s orientation and facilitate accurate turns.
  • 1 Color Sensor: Employed to detect specific markers on the course, ensuring the robot positioned itself correctly for ball launching.

Ball Bearings:

  • Used within the wheel assembly, these components reduced friction and enhanced the robot’s mobility and stability, this was crucial for its movement and positioning accuracy.

RoboHacks 2024 🚗

The Robotics City Sweep Challenge involves a robot tasked with cleaning and sorting various types of waste—recycling, compost, and garbage—scattered across a playing field into appropriate bins. The challenge tests the robot’s ability to autonomously navigate diverse terrains, identify and collect waste, and sort it accurately within a 3-minute round, with a potential 30-second extension

We implemented the wireless control using ESP32 microcontroller programmed with C++ firmware, enabling remote operation via Wi-Fi laptop connectivity.

Collection System

Purpose: The collection system’s main goal was to gather and sort waste found throughout the playing field effectively. It had to handle different types of waste and ensure correct deposition in the corresponding waste bins to maximize points.

Motors:

  • 1 Servo Motor: Utilized for precise control of the waste collection mechanism, allowing the robot to pick up and sort waste materials accurately.

Software Overview

The robot’s operations were governed by software written in C++, with the exception of using a Python script for the WIFI connection between the robot and the laptop.

Navigation System

Purpose: The navigation system enabled the robot to move strategically across different zones of the playing field, which include residential areas, mountains, forests, and downtown regions, each with unique navigation challenges.

Motors:

  • 2 Power Motors: Provided the necessary propulsion for the robot to move through the different terrains and obstacles presented in the challenge.

Sensors and Additional Components:

  • 2 Rubber Wheels and 1 Ball Bearing: Allowed for maneuverability and stability as the robot moved through various terrains and over obstacles.

Back to the top