Flappy Es

Explore resources related to flappy es to help implement AI solutions for your business.

From "There's an App for that" to "There's YOUR App for that" - AI workflows will transform generic apps into deeply personalized experiences

From "There's an App for that" to "There's YOUR App for that" - AI workflows will transform generic apps into deeply personalized experiences

I will not promote. For the past decade mobile apps were a core element of daily life for entertainment, productivity and connectivity. However, as the ecosystem saturated the general desire to download "just one more app" became apprehensive. There were clear monopolistic winners in different categories, such as Instagram and TikTok, which completely captured the majority of people's screentime. The golden age of creating indie apps and becoming a millionaire from them was dead. Conceptual models of these popular apps became ingrained in the general consciousness, and downloading new apps where re-learning new UI layouts was required, became a major friction point. There is high reluctance to download a new app rather than just utilizing the tooling of the growing market share of the existing winners. Content marketing and white labeled apps saw a resurgence of new app downloads, as users with parasympathetic relationships with influencers could be more easily persuaded to download them. However, this has led to a series of genericized tooling that lacks the soul of the early indie developer apps from the 2010s (Flappy bird comes to mind). A seemingly grim spot to be in, until everything changed on November 30th 2022. Sam Altman, Ilya Sutskever and team announced chatGPT, a Large Language Model that was the first publicly available generative AI tool. The first non-deterministic tool that could reason probablisitically in a similar (if flawed) way, to the human mind. At first, it was a clear paradigm shift in the world of computing, this was obvious from the fact that it climbed to 1 Million users within the first 5 days of its launch. However, despite the insane hype around the AI, its utility was constrained to chatbot interfaces for another year or more. As the models reasoning abilities got better and better, engineers began to look for other ways of utilizing this new paradigm shift, beyond chatbots. It became clear that, despite the powerful abilities to generate responses to prompts, the LLMs suffered from false hallucinations with extreme confidence, significantly impacting the reliability of their use, in search, coding and general utility. Retrieval Augmented Generation (RAG) was coined to provide a solution to this. Now, the LLM would apply a traditional search for data, via a database, a browser or other source of truth, and then feed that information into the prompt as it generates, allowing for more accurate results. Furthermore, it became clear that you could enhance an LLM by providing them metadata to interact with tools such as APIs for other services, allowing LLMs to perform actions typically reserved for humans, like fetching data, manipulating it and acting as an independent Agent. This prompted engineers to start treating LLMs, not as a database and a search engine, but rather a reasoning system, that could be part of a larger system of inputs and feedback to handle workflows independently. These "AI Agents" are poised to become the core technology in the next few years for hyper-personalizing and automating processes for specific users. Rather than having a generic B2B SaaS product that is somewhat useful for a team, one could standup a modular system of Agents that can handle the exactly specified workflow for that team. Frameworks such as LlangChain and LLamaIndex will help enable this for companies worldwide. The power is back in the hands of the people. However, it's not just big tech that is going to benefit from this revolution. AI Agentic workflows will allow for a resurgence in personalized applications that work like personal digital employee's. One could have a Personal Finance agent keeping track of their budgets, a Personal Trainer accountability coaching you making sure you meet your goals, or even a silly companion that roasts you when you're procrastinating. The options are endless ! At the core of this technology is the fact that these agents will be able to recall all of your previous data and actions, so they will get better at understanding you and your needs as a function of time. We are at the beginning of an exciting period in history, and I'm looking forward to this new period of deeply personalized experiences. What are your thoughts ? Let me know in the comments !

#reddit #"there's #that"

flappy-es

Playing Flappy Bird using Evolution Strategies ============================================== After reading Evolution Strategies as a Scalable Alternative to Reinforcement Learning, I wanted to experiment something using Evolution Strategies, and Flappy Bird has always been one of my favorites when it comes to Game experiments. A simple yet challenging game. The model learns to play very well after 3000 epochs, but not completely flawless and it rarely loses in difficult cases (high difference between two wall entrances). Training process is pretty fast as there is no backpropagation, and is not very costy in terms of memory as there is no need to record actions as in policy gradients. Here is a demonstration of the model after 3000 epochs (~5 minutes on an Intel(R) Core(TM) i7-4770HQ CPU @ 2.20GHz): !after training Before training: !Before training There is also a a web version available for ease of access. For each frame the bird stays alive, +0.1 score is given to him. For each wall he passes, +10 score is given. Demonstration of rewards for individuals and the mean reward over time (y axis is logarithmic): !reward chart Try it yourself You need python3.5 and pip for installing and running the code. First, install dependencies (you might want to create a virtualenv): The pretrained parameters are in a file named load.npy and will be loaded when you run train.py or demo.py. train.py will train the model, saving the parameters to saves//save-. demo.py shows the game in a GTK window so you can see how the AI actually plays (like the GIF above). play.py if you feel like playing the game yourself, space: jump, once lost, press enter to play again. :grin: pro tip: reach 100 score and you will become THUG FOR LIFE :smoking: Notes It seems training past a maximum point reduces performance, learning rate decay might help with that. My interpretation is that after finding a local maximum for accumulated reward and being able to receive high rewards, the updates become pretty large and will pull the model too much to sides, thus the model will enter a state of oscillation. To try it yourself, there is a long.npy file, rename it to load.npy (backup load.npy before doing so) and run demo.py, you will see the bird failing more often than not. long.py was trained for only 100 more epochs than load.npy.

alirezamika•Mar 23, 2025

evostra

Evostra: Evolution Strategy for Python Evolution Strategy (ES) is an optimization technique based on ideas of adaptation and evolution. You can learn more about it at https://blog.openai.com/evolution-strategies/ Installation It's compatible with both python2 and python3. Install from source: .. code-block:: bash $ python setup.py install Install latest version from git repository using pip: .. code-block:: bash $ pip install git+https://github.com/alirezamika/evostra.git Install from PyPI: .. code-block:: bash $ pip install evostra (You may need to use python3 or pip3 for python3) Sample Usages An AI agent learning to play flappy bird using evostra _ An AI agent learning to walk using evostra _ How to use The input weights of the EvolutionStrategy module is a list of arrays (one array with any shape for each layer of the neural network), so we can use any framework to build the model and just pass the weights to ES. For example we can use Keras to build the model and pass its weights to ES, but here we use Evostra's built-in model FeedForwardNetwork which is much faster for our use case: .. code:: python import numpy as np from evostra import EvolutionStrategy from evostra.models import FeedForwardNetwork A feed forward neural network with input size of 5, two hidden layers of size 4 and output of size 3 model = FeedForwardNetwork(layer_sizes=[5, 4, 4, 3]) Now we define our get_reward function: .. code:: python solution = np.array([0.1, -0.4, 0.5]) inp = np.asarray([1, 2, 3, 4, 5]) def get_reward(weights): global solution, model, inp model.set_weights(weights) prediction = model.predict(inp) here our best reward is zero reward = -np.sum(np.square(solution - prediction)) return reward Now we can build the EvolutionStrategy object and run it for some iterations: .. code:: python if your task is computationally expensive, you can use num_threads > 1 to use multiple processes; if you set num_threads=-1, it will use number of cores available on the machine; Here we use 1 process as the task is not computationally expensive and using more processes would decrease the performance due to the IPC overhead. es = EvolutionStrategy(model.getweights(), getreward, populationsize=20, sigma=0.1, learningrate=0.03, decay=0.995, num_threads=1) es.run(1000, print_step=100) Here's the output: .. code:: iter 100. reward: -68.819312 iter 200. reward: -0.218466 iter 300. reward: -0.110204 iter 400. reward: -0.001901 iter 500. reward: -0.000459 iter 600. reward: -0.000287 iter 700. reward: -0.000939 iter 800. reward: -0.000504 iter 900. reward: -0.000522 iter 1000. reward: -0.000178 Now we have the optimized weights and we can update our model: .. code:: python optimizedweights = es.getweights() model.setweights(optimizedweights) Todo Add distribution support over network

#github #evostra