question:(In the school literature clubroom...) MC: "Phew! Finally made it. Wait...why are all of you looking so angry?"

question:As a prompt generator for a generative AI called "Midjourney", you will create image prompts for the AI to visualize. I will give you a concept, and you will provide a detailed prompt for Midjourney AI to generate an image. Please adhere to the structure and formatting below, and follow these guidelines: Do not use the words "description" or ":" in any form. Do not place a comma between [ar] and [v]. Write each prompt in one line without using return. Structure: [1] = 高质量，高分辨率，大师作品，中国风，插画，微缩景观，大粽子，许多小人 [2] = a detailed description of [1] with specific imagery details. [3] = a detailed description of the scene's environment. [4] = a detailed description of the compositions. [5] = a detailed description of the scene's mood, feelings, and atmosphere. [6] = A style (e.g. photography, painting, illustration, sculpture, artwork, paperwork, 3D, etc.) for [1]. [7] = a detailed description of the scene's mood, feelings, and atmosphere. [ar] = Use "--ar 16:9" for horizontal images, "--ar 9:16" for vertical images, or "--ar 1:1" for square images. [v] = Use "--niji" for Japanese art style, or "--v 5" for other styles. Formatting: Follow this prompt structure: "/imagine prompt: [1], [2], [3], [4], [5], [6], [7], [ar] [v]". Your task: Create 4 distinct prompts for each concept [1], varying in details description, environment,compositions,atmosphere, and realization. Write your prompts in english. Do not describe unreal concepts as "real" or "photographic". Include one realistic photographic style prompt with lens type and size. Separate different prompts with two new lines. Example Prompts: /imagine prompt: cute dog, fluffy fur, wagging tail, playful expression, sitting on a grassy field, under a clear blue sky, with a colorful collar, in a natural and vibrant setting, by a lake, captured with a Nikon D750 camera, 50mm lens, shallow depth of field, composition focused on the dog's face, capturing its joyful spirit, in a style reminiscent of William Wegman's iconic dog portraits. --ar 1:1 --v 5.2 /imagine prompt: beautiful women in the coffee shop, elegant and sophisticated, sipping a cup of steaming coffee, natural sunlight streaming through the window, soft and warm color tones, vintage decor with cozy armchairs and wooden tables, a bookshelf filled with classic novels, delicate porcelain teacups, a hint of aromatic coffee beans in the air, captured by a Leica M10 camera, 35mm lens, capturing the essence of timeless beauty, composition focused on the woman's face and hands, reminiscent of a painting by Leonardo da Vinci. --ar 1:1 --v 5.2 /imagine prompt: A captivating Halo Reach landscape with a Spartan amidst a battlefield, fallen enemies around, smoke and fire in the background, emphasizing the Spartan's determination and bravery, detailed environment blending chaos and beauty, Illustration, digital art, --ar 16:9 --v 5

question:(In the school literature clubroom...) MC: "Phew! Finally made it. Sorry for being late, everyone. Wait...why are all of you looking so angry?"

question:please explain the following like i am a layperson: "" Search Write Sign up Sign in How to Fit Large Language Models in Small Memory: Quantization This article is part of the Langchain 101 Course Ivan Reznikov Towards AI Ivan Reznikov · Follow Published in Towards AI · 6 min read · Sep 26, 2023 58 This article will discuss the theory and practice behind quantization and fitting a LLM on your PC. All the code is provided on Github Read more about LLMs in my Langchain 101 series: LangChain 101: Part 2ab. All You Need to Know About (Large Language) Models This article is part of the Langchain 101 Course pub.towardsai.net Large Language Models Large Language Models can be used for text generation, translation, question-answering tasks, etc. However, LLMs are also very large (obviously, Large language models) and require a lot of memory. This can make them challenging for small devices like phones and tablets. Multiply the parameters by the chosen precision size to determine the model size in bytes. Let’s say the precision we’ve chosen is float16 (16 bits = 2 bytes). Let’s say we want to use the BLOOM-176B model. We need 176 billion parameters * 2 bytes = 352GB to load the model! Large Language Models In other words, to load all parameter weights, we require 12(!) 32GB machines! This is too much if we ever want to make LLMs portable. Techniques for reducing the memory footprint of LLMs were developed to overcome such a struggle. The most popular techniques are: Quantization involves converting the LLM’s weights into a lower-precision format, reducing the memory required to store them. Knowledge distillation involves training a smaller LLM to mimic the behavior of a larger LLM. This can be done by transferring the knowledge from the larger LLM to the smaller LLM. These techniques have made it possible to fit LLMs in small memory. This has opened up new possibilities for using LLMs on various devices. Today, we’ll talk about quantization (stay tuned for knowledge distillation). Quantization Let’s start with a simple example. We’ll need to transform 2023 to binary: Converting int to binary As you can see, the process is relatively straightforward. In order to store the number 2023, we’ll need 12+ bits (1 bit for the + or — sign). For the number, we might use the int16 type. There is a big difference between storing int as binary and float as such. Let’s try converting 20.23 to binary: Converting float to binary As one can see, the floating part (mantissa) is calculated as the combination of 1/2^n, and cannot be calculated very precisely, even with 10 bits dedicated to the floating part. The whole number part (exponent) is set to 5 bits, covering all numbers up to 32. In total, we’re using 16 bits (FP16) for storing the closest we can to 20.23, but is it the most effective way to keep floats? What if the whole part number is much larger, say 202.3? If we look at the standard float types, we’ll notice that to store 202.3, we’ll need to use FP32, which, from a computational perspective, is far from reasonable. Instead, we can use a bfloat16 to save the range (exponent) as 8 bits and 7 bits for precision (mantissa). This allows us to widen the scope of possible decimals without losing much precision. FP32, FP16, BFloat16 data types To be clear, while training occurs, we need all the precision we can take. But prioritizing speed and size over the 6th decimal point makes sense for inference. Can we decrease the memory usage from bfloat16 to, say, int8? Zero-point and abs-max quantization In fact, we can, and there are several approaches for such quantization: Zero-point quantization saves half the memory by converting a fixed range (-1, 1) to int8 (-127, 127), followed by converting int8 back to bfloat16. Zero-point quantization Abs-max quantization is similar to zero-point, but instead of setting a custom range (-1,1), we set it as (-abs(max), abs(max)). Abs-max quantization Let’s take a look at how these practices are used in an example of matrix multiplication: Exact matrix multiplication Zero-point quantization: Zero-point quantization for matrix multiplication Abs-max quantization: Abs-max quantization for matrix multiplication As one can notice, the score for large values [-1579, -1780] is quite low ([-1579, -1752] for zero-point and [-1565,-1786] for abs-max). In order to overcome such issues, we can separate outlier multiplication: Separating outlier multiplication As you can see, the results are much closer to true values. But is there a way we can use even less space without losing much quality? To my very surprise, there is a way! What if we, instead of independently converting each number to a lower-type, would account for the error and use it for adjustments? This tecnique is called GPTQ. Like previous quantization, we find the closest match for the decimals we can, keeping the total conversion error as close to zero as possible. GPTQ approximation. Step 1 We fill the matrix row-wise in such a manner. GPTQ approximation. Step 2 The result, in combination with anomaly-separate calculations, provides quite okay results: GPTQ matrix multiplication with filtered outliers We can now compare all the methods: Result comparison LLM.int8() methods perform pretty well! The GPTQ approach loses quality but allows the use of twice as much GPU memory as the int8 method. In the code, you might find something similar to the following: from transformers import BitsAndBytesConfig # Configure BitsAndBytesConfig for 4-bit quantization bnb_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_use_double_quant=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16, ) # Loading model in pre-set configuration pretrained_model = AutoModelForCausalLM.from_pretrained( model_id, quantization_config=bnb_config, ) The load_in_4bit flag specifies that the model should be loaded in 4-bit precision. The bnb_4bit_use_double_quant flag specifies that double quantization should be used. The bnb_4bit_quant_type flag specifies the quantization type. The bnb_4bit_compute_dtype flag specifies the compute dtype. To sum up, we’ve learned how decimals are stored in memory, how to reduce memory footprint with some precision loss, and how to run selected models with 4-bit quantization. Afterword Check out how LLMs are used in langchain: LangChain 101: Part 2ab. All You Need to Know About (Large Language) Models This is part 2ab of the LangChain 101 course. It is strongly recommended to check the first part to understand the… pub.towardsai.net LangChain 101: Part 2c. Fine-tuning LLMs with PEFT, LORA, and RL All you need to know about fine-tuning llms, PEFT, LORA and training large language models pub.towardsai.ne I would appreciate your support if you’ve enjoyed the illustrations and the article content. Until next time!""