整合營銷服務商

          電腦端+手機端+微信端=數據同步管理

          免費咨詢熱線:

          這一年,NLP突破進展真不少:BERT變體遍出,紀錄

          這一年,NLP突破進展真不少:BERT變體遍出,紀錄一破再破

          三 發自 凹非寺
          量子位 報道 | 公眾號 QbitAI

          2019年,自然語言處理(NLP)都取得了哪些突破?

          提到NLP,BERT可以說是家喻戶曉。

          在情感分析、問答、句子相似度等多個 NLP 任務上都取得了優異的成績。

          而且,無論是在類似于Kaggle這樣的競賽,或者媒體報道中,也總能看到它的身影。

          它發表于2018年末,自那之后的一年,NLP和NLU(自然語言理解)領域有了較大的發展。

          那么,以BERT的發布作為時間節點,本文便梳理了一下在此之前和之后,NLP領域的重要項目和模型。

          BERT之前的一些主要 NLP 項目時間表

          在提出BERT模型之前,NLP領域中的主要項目按時間排序,如下圖所示:

          Word2Vec模型發布于2013年1月,至今也是非常流行。

          在任何NLP任務中,研究人員可能嘗試的第一個模型就是它。

          https://arxiv.org/abs/1301.3781

          FastTextGloVe分別于2016年7月和2014年1月提出。

          FastText是一個開源的、免費的、輕量級的庫,它允許用戶學習文本表示和文本分類器。

          https://fasttext.cc/

          GloVe是一種無監督的學習算法,用于獲取單詞的向量表示。

          https://nlp.stanford.edu/projects/glove/

          Transformer于2017年6月提出,是一種基于 encoder-decoder 結構的模型。

          在機器翻譯任務上的表現超過了 RNN,CNN,只用 encoder-decoder 和 attention 機制就能達到很好的效果,最大的優點是可以高效地并行化。

          https://ai.googleblog.com/2017/08/transformer-novel-neural-network.html

          ELMo于2018年2月提出,利用預訓練好的雙向語言模型,然后根據具體輸入從該語言模型中可以得到上下文依賴的當前詞表示,再當成特征加入到具體的NLP有監督模型里。

          https://allennlp.org/elmo

          還有一個叫Ulmfit,是面向NLP任務的遷移學習模型,只需使用極少量的標記數據,文本分類精度就能和數千倍的標記數據訓練量達到同等水平。

          https://arxiv.org/abs/1801.06146

          值得注意的是,ELMo和Ulmfit出現在BERT之前,沒有采用基于Transformer的結構。

          BERT

          BERT模型于2018年10月提出。

          全稱是Bidirectional Encoder Representation from Transformers,即雙向Transformer的Encoder(因為decoder不能獲取要預測的信息)。

          △論文地址:https://arxiv.org/abs/1810.04805

          模型的主要創新點都在pre-train方法上,即用了Masked LM和Next Sentence Prediction兩種方法分別捕捉詞語和句子級別的表示。

          谷歌甚至開始使用BERT來改善搜索結果。

          奉上一份較為詳細的BERT模型教程:
          http://jalammar.github.io/illustrated-bert/

          預訓練權重相關內容可以從官方 Github repo 下載:
          https://github.com/google-research/bert

          Bert 也可以作為 Tensorflow hub 模塊:
          https://tfhub.dev/google/collections/bert/1

          文末還會奉上各種非常實用的庫。

          BERT之后的一些主要 NLP 項目時間表

          在谷歌提出BERT之后,NLP領域也相繼出了其他較為突出的工作項目。

          Transformer-XL

          Transormer-XL是Transformer的升級版,在速度方面比Transformer快1800多倍。

          這里的XL,指的是extra long,意思是超長,表示Transformer-XL在語言建模中長距離依賴問題上有非常好的表現。同時,也暗示著它就是為長距離依賴問題而生。

          長距離依賴問題,是當前文本處理模型面臨的難題,也是RNN失敗的地方。

          相比之下,Transformer-XL學習的依賴要比RNN長80%。比Vanilla Transformers快450%。

          在短序列和長序列上,都有很好的性能表現。

          https://arxiv.org/abs/1901.02860

          GPT-2

          GPT-2可以說是在BERT之后,媒體報道最為關注的一個NLP模型。

          這是OpenAI發布的一個“逆天”的語言AI,整個模型包含15億個參數。

          無需針對性訓練就能橫掃各種特定領域的語言建模任務,還具備閱讀理解、問答、生成文章摘要、翻譯等等能力。

          而且,OpenAI最初還擔心項目過于強大,而選擇沒有開源。但在10個月之后,還是決定將其公布。

          https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf

          ERNIE

          ERNIE是基于百度自己的深度學習框架飛槳(PaddlePaddle)搭建的,可以同時利用詞匯、句法和知識信息。

          實驗結果顯示,在不同的知識驅動任務取得了顯著的改進,同時在其它常見任務上與現有的BERT模型具有可比性。

          當前,ERNIE 2.0版本在GLUE排行榜上排名第一。
          https://github.com/PaddlePaddle/ERNIE

          XLNET

          XLNet 是一個類似BERT的模型,是一種通用的自回歸預訓練方法。

          它不使用傳統 AR 模型中固定的前向或后向因式分解順序,而是最大化所有可能因式分解順序的期望對數似然。

          其次,作為一個泛化 AR 語言模型,XLNet不依賴殘缺數據。

          此外,XLNet還改進了預訓練的架構設計。

          https://arxiv.org/abs/1906.08237

          RoBERTa

          RoBERTa由Facebook提出。

          它在模型層面沒有改變谷歌的BERT,改變的只是預訓練的方法。

          在模型規模、算力和數據上,與BERT相比主要有以下幾點改進:

          更大的模型參數量:模型使用 1024 塊 V100 GPU 訓練了 1 天的時間。

          更大bacth size:RoBERTa在訓練過程中使用了更大的bacth size,嘗試過從 256 到 8000 不等的bacth size。

          更多的訓練數據:包括CC-NEWS 等在內的160GB純文本。

          https://arxiv.org/abs/1907.11692

          Salesforce CTRL

          CTRL全名是Conditional Transformer Language,包含16億個參數。

          它具有強大且可控的人工文本生成功能,可以預測哪個訓練數據子集對生成的文本序列影響最大。

          通過識別模型中最有影響力的訓練數據來源,為分析大量生成的文本提供了一種潛在的方法。

          CTRL還可以通過微調特定任務或轉移模型已學習的表示形式來改進其他NLP應用程序。

          https://blog.einstein.ai/introducing-a-conditional-transformer-language-model-for-controllable-generation/

          ALBERT

          ALBERT是谷歌發布的輕量級BERT模型。

          比BERT模型參數小18倍,性能還超越了它,在SQuAD和RACE測試上創造了新的SOTA。

          前不久,谷歌還對此進行了升級,發布了ALBERT 2和中文版本。

          在這個版本中,“no dropout”、“additional training data”、“long training time”策略將應用到所有的模型。

          從性能的比較來說,對于ALBERT-base、ALBERT-large和ALBERT-xlarge,v2版要比v1版好得多。

          說明采用上述三個策略的重要性。

          https://arxiv.org/abs/1909.11942

          性能評測基準

          評估這些語言模型的方法之一是Glue Benchmark

          它包括評估模型的各種NLP任務,如分類、問答等。

          在Glue Benchmark剛剛發布的時候,BERT模型的性能位居榜首。

          但截至2020年1月2日,在僅僅1年時間內,BERT已經排名到了19位。

          現在還有一個 SuperGlue 基準測試,它包含了更難理解的語言任務。

          對于評估問題回答系統,SQuAD是較為常用的。

          BERT和基于transformer模型在此處的性能是較好的。

          其它與BERT相關項目

          DistilBERT

          DistilBERT是HuggingFace發布的小型NLP transformer模型,與BERT的架構類似,不過它僅使用了 6600 萬參數,但在 GLUE 基準上實現了BERT 95% 的性能。

          https://arxiv.org/abs/1910.01108

          Megatron-LM

          Megatron-LM是英偉達發布的NLP模型。

          英偉達用自己的硬件與并行計算軟件相結合,當時創下了三項紀錄:

          訓練速度只需53分鐘;
          推理速度只需2.2ms;
          包含83億參數。

          https://github.com/NVIDIA/Megatron-LM

          BioBERT

          BioBERT是用于生物醫學文本挖掘的預訓練生物醫學語言表示模型。

          在生物醫學語料庫上進行預培訓時,它在各種生物醫學文本挖掘任務上的表現,在很大程度上超過了BERT和之前的先進模型。

          https://github.com/dmis-lab/biobert

          CamemBERT

          CamemBERT是一種基于RoBERTa 結構的法語語言模型。

          https://camembert-model.fr/

          NLP庫

          下面是作者認為需要了解的一些NLP庫。

          Spacy

          Spacy 是一個流行的、快速的NLP程序庫,可以處理各種自然語言處理任務,如標記、詞性等。它還提供了預先訓練的NER等模型。

          https://spacy.io/

          HuggingFace Transformers

          它是首批提供 BERT Pytorch實現的庫之一,最初被稱為“ Pytorch-pretrained-BERT”。

          后來,他們增加了更多的模型,如GPT-2,XLNET等。

          在不到一年的時間里,它已經成為最流行的 NLP 庫之一,并且使得BERT和其他模型的使用變得更加容易。

          https://github.com/huggingface/transformers

          AllenNLP

          AllenNLP是來自艾倫人工智能研究所(Allen Institute of AI)的NLP庫,基于PyTorch。

          https://allennlp.org/

          Flair

          Flair也是一個帶有 NER、 POS 等模型的 NLP 庫,還支持 BERT、 ELMO、 XLNET 等嵌入。

          https://github.com/flairNLP/flair

          GluonNLP

          GluonNLP是Apache MXNet 上的NLP工具包,是最早包含預先訓練的BERT嵌入式的庫之一。

          https://gluon-nlp.mxnet.io/

          那么,在2020年,NLP又會怎樣的突破呢?

          傳送門

          https://towardsdatascience.com/2019-year-of-bert-and-transformer-f200b53d05b9

          — 完 —

          量子位 QbitAI · 頭條號簽約

          關注我們,第一時間獲知前沿科技動態

          篇文章主要列舉了第三人稱的多種控制方式。

          一、官方實例的第三人稱控制方式。

          該控制方式比較復雜,但是卻寫得很好很完善,并且運用了新的動畫系統。大家可以下載官方的角色控制包來使用,附上圖一張,不多說。

          二、老版官方的第三人稱控制方式。

          大家應該知道老版的第三人稱控制方式是用JavaScript腳本寫的,可能大家拿過來還不太好用,但是這里我們把它改寫成C#腳本(PS:參照雨松的修改),這樣用起來就方便多了,而且用的是經典版的動畫系統,滿足了很多人的需求。

          在unity中,新版的mecanim動畫系統出現,雖然說很實用,在某些方面解決了很多人的需求,但這并不意味著可以替代原版經典的動畫系統,所以到現在為止,兩種動畫都是通用的。

          using UnityEngine;

          using System.Collections;

          [RequireComponent(typeof(CharacterController))]

          public class ThirdPersonController111 : MonoBehaviour

          {

          public AnimationClip idleAnimation;

          public AnimationClip walkAnimation;

          public AnimationClip runAnimation;

          public AnimationClip jumpPoseAnimation;

          public float walkMaxAnimationSpeed=0.75f;

          public float trotMaxAnimationSpeed=1.0f;

          public float runMaxAnimationSpeed=1.0f;

          public float jumpAnimationSpeed=1.15f;

          public float landAnimationSpeed=1.0f;

          private Animation _animation;

          enum CharacterState

          {

          Idle=0,

          Walking=1,

          Trotting=2,

          Running=3,

          Jumping=4,

          }

          private CharacterState _characterState;

          // The speed when walking

          float walkSpeed=2.0f;

          // after trotAfterSeconds of walking we trot with trotSpeed

          float trotSpeed=4.0f;

          // when pressing "Fire3" button (cmd) we start running

          float runSpeed=6.0f;

          float inAirControlAcceleration=3.0f;

          // How high do we jump when pressing jump and letting go immediately

          float jumpHeight=0.5f;

          // The gravity for the character

          float gravity=20.0f;

          // The gravity in controlled descent mode

          float speedSmoothing=10.0f;

          float rotateSpeed=500.0f;

          float trotAfterSeconds=3.0f;

          bool canJump=true;

          private float jumpRepeatTime=0.05f;

          private float jumpTimeout=0.15f;

          private float groundedTimeout=0.25f;

          // The camera doesnt start following the target immediately but waits for a split second to avoid too much waving around.

          private float lockCameraTimer=0.0f;

          // The current move direction in x-z

          private Vector3 moveDirection=Vector3.zero;

          // The current vertical speed

          private float verticalSpeed=0.0f;

          // The current x-z move speed

          private float moveSpeed=0.0f;

          // The last collision flags returned from controller.Move

          private CollisionFlags collisionFlags;

          // Are we jumping? (Initiated with jump button and not grounded yet)

          private bool jumping=false;

          private bool jumpingReachedApex=false;

          // Are we moving backwards (This locks the camera to not do a 180 degree spin)

          private bool movingBack=false;

          // Is the user pressing any keys?

          private bool isMoving=false;

          // When did the user start walking (Used for going into trot after a while)

          private float walkTimeStart=0.0f;

          // Last time the jump button was clicked down

          private float lastJumpButtonTime=-10.0f;

          // Last time we performed a jump

          private float lastJumpTime=-1.0f;

          // the height we jumped from (Used to determine for how long to apply extra jump power after jumping.)

          private float lastJumpStartHeight=0.0f;

          private Vector3 inAirVelocity=Vector3.zero;

          private float lastGroundedTime=0.0f;

          private bool isControllable=true;

          void Awake()

          {

          moveDirection=transform.TransformDirection(Vector3.forward);

          _animation=GetComponent<Animation>();

          if (!_animation)

          Debug.Log("The character you would like to control doesn't have animations. Moving her might look weird.");

          /*

          public var idleAnimation : AnimationClip;

          public var walkAnimation : AnimationClip;

          public var runAnimation : AnimationClip;

          public var jumpPoseAnimation : AnimationClip;

          */

          if (!idleAnimation)

          {

          _animation=null;

          Debug.Log("No idle animation found. Turning off animations.");

          }

          if (!walkAnimation)

          {

          _animation=null;

          Debug.Log("No walk animation found. Turning off animations.");

          }

          if (!runAnimation)

          {

          _animation=null;

          Debug.Log("No run animation found. Turning off animations.");

          }

          if (!jumpPoseAnimation && canJump)

          {

          _animation=null;

          Debug.Log("No jump animation found and the character has canJump enabled. Turning off animations.");

          }

          }

          void UpdateSmoothedMovementDirection()

          {

          Transform cameraTransform=Camera.main.transform;

          bool grounded=IsGrounded();

          // Forward vector relative to the camera along the x-z plane

          Vector3 forward=cameraTransform.TransformDirection(Vector3.forward);

          forward.y=0;

          forward=forward.normalized;

          // Right vector relative to the camera

          // Always orthogonal to the forward vector

          Vector3 right=new Vector3(forward.z, 0, -forward.x);

          float v=Input.GetAxisRaw("Vertical");

          float h=Input.GetAxisRaw("Horizontal");

          // Are we moving backwards or looking backwards

          if (v < -0.2f)

          movingBack=true;

          else

          movingBack=false;

          bool wasMoving=isMoving;

          isMoving=Mathf.Abs(h) > 0.1f || Mathf.Abs(v) > 0.1f;

          // Target direction relative to the camera

          Vector3 targetDirection=h * right + v * forward;

          // Grounded controls

          if (grounded)

          {

          // Lock camera for short period when transitioning moving & standing still

          lockCameraTimer +=Time.deltaTime;

          if (isMoving !=wasMoving)

          lockCameraTimer=0.0f;

          // We store speed and direction seperately,

          // so that when the character stands still we still have a valid forward direction

          // moveDirection is always normalized, and we only update it if there is user input.

          if (targetDirection !=Vector3.zero)

          {

          // If we are really slow, just snap to the target direction

          if (moveSpeed < walkSpeed * 0.9f && grounded)

          {

          moveDirection=targetDirection.normalized;

          }

          // Otherwise smoothly turn towards it

          else

          {

          moveDirection=Vector3.RotateTowards(moveDirection, targetDirection, rotateSpeed * Mathf.Deg2Rad * Time.deltaTime, 1000);

          moveDirection=moveDirection.normalized;

          }

          }

          // Smooth the speed based on the current target direction

          float curSmooth=speedSmoothing * Time.deltaTime;

          // Choose target speed

          //* We want to support analog input but make sure you cant walk faster diagonally than just forward or sideways

          float targetSpeed=Mathf.Min(targetDirection.magnitude, 1.0f);

          _characterState=CharacterState.Idle;

          // Pick speed modifier

          if (Input.GetKey(KeyCode.LeftShift) | Input.GetKey(KeyCode.RightShift))

          {

          targetSpeed *=runSpeed;

          _characterState=CharacterState.Running;

          }

          else if (Time.time - trotAfterSeconds > walkTimeStart)

          {

          targetSpeed *=trotSpeed;

          _characterState=CharacterState.Trotting;

          }

          else

          {

          targetSpeed *=walkSpeed;

          _characterState=CharacterState.Walking;

          }

          moveSpeed=Mathf.Lerp(moveSpeed, targetSpeed, curSmooth);

          // Reset walk time start when we slow down

          if (moveSpeed < walkSpeed * 0.3f)

          walkTimeStart=Time.time;

          }

          // In air controls

          else

          {

          // Lock camera while in air

          if (jumping)

          lockCameraTimer=0.0f;

          if (isMoving)

          inAirVelocity +=targetDirection.normalized * Time.deltaTime * inAirControlAcceleration;

          }

          }

          void ApplyJumping()

          {

          // Prevent jumping too fast after each other

          if (lastJumpTime + jumpRepeatTime > Time.time)

          return;

          if (IsGrounded())

          {

          // Jump

          // - Only when pressing the button down

          // - With a timeout so you can press the button slightly before landing

          if (canJump && Time.time < lastJumpButtonTime + jumpTimeout)

          {

          verticalSpeed=CalculateJumpVerticalSpeed(jumpHeight);

          SendMessage("DidJump", SendMessageOptions.DontRequireReceiver);

          }

          }

          }

          void ApplyGravity()

          {

          if (isControllable) // don't move player at all if not controllable.

          {

          // Apply gravity

          bool jumpButton=Input.GetButton("Jump");

          // When we reach the apex of the jump we send out a message

          if (jumping && !jumpingReachedApex && verticalSpeed <=0.0f)

          {

          jumpingReachedApex=true;

          SendMessage("DidJumpReachApex", SendMessageOptions.DontRequireReceiver);

          }

          if (IsGrounded())

          verticalSpeed=0.0f;

          else

          verticalSpeed -=gravity * Time.deltaTime;

          }

          }

          float CalculateJumpVerticalSpeed(float targetJumpHeight)

          {

          // From the jump height and gravity we deduce the upwards speed

          // for the character to reach at the apex.

          return Mathf.Sqrt(2 * targetJumpHeight * gravity);

          }

          void DidJump()

          {

          jumping=true;

          jumpingReachedApex=false;

          lastJumpTime=Time.time;

          lastJumpStartHeight=transform.position.y;

          lastJumpButtonTime=-10;

          _characterState=CharacterState.Jumping;

          }

          void Update()

          {

          if (!isControllable)

          {

          // kill all inputs if not controllable.

          Input.ResetInputAxes();

          }

          if (Input.GetButtonDown("Jump"))

          {

          lastJumpButtonTime=Time.time;

          }

          UpdateSmoothedMovementDirection();

          // Apply gravity

          // - extra power jump modifies gravity

          // - controlledDescent mode modifies gravity

          ApplyGravity();

          // Apply jumping logic

          ApplyJumping();

          // Calculate actual motion

          Vector3 movement=moveDirection * moveSpeed + new Vector3(0, verticalSpeed, 0) + inAirVelocity;

          movement *=Time.deltaTime;

          // Move the controller

          CharacterController controller=GetComponent<CharacterController>();

          collisionFlags=controller.Move(movement);

          // ANIMATION sector

          if (_animation)

          {

          if (_characterState==CharacterState.Jumping)

          {

          if (!jumpingReachedApex)

          {

          _animation[jumpPoseAnimation.name].speed=jumpAnimationSpeed;

          _animation[jumpPoseAnimation.name].wrapMode=WrapMode.ClampForever;

          _animation.CrossFade(jumpPoseAnimation.name);

          }

          else

          {

          _animation[jumpPoseAnimation.name].speed=-landAnimationSpeed;

          _animation[jumpPoseAnimation.name].wrapMode=WrapMode.ClampForever;

          _animation.CrossFade(jumpPoseAnimation.name);

          }

          }

          else

          {

          if (controller.velocity.sqrMagnitude < 0.1f)

          {

          _animation.CrossFade(idleAnimation.name);

          }

          else

          {

          if (_characterState==CharacterState.Running)

          {

          _animation[runAnimation.name].speed=Mathf.Clamp(controller.velocity.magnitude, 0.0f, runMaxAnimationSpeed);

          _animation.CrossFade(runAnimation.name);

          }

          else if (_characterState==CharacterState.Trotting)

          {

          _animation[walkAnimation.name].speed=Mathf.Clamp(controller.velocity.magnitude, 0.0f, trotMaxAnimationSpeed);

          _animation.CrossFade(walkAnimation.name);

          }

          else if (_characterState==CharacterState.Walking)

          {

          _animation[walkAnimation.name].speed=Mathf.Clamp(controller.velocity.magnitude, 0.0f, walkMaxAnimationSpeed);

          _animation.CrossFade(walkAnimation.name);

          }

          }

          }

          }

          // ANIMATION sector

          // Set rotation to the move direction

          if (IsGrounded())

          {

          transform.rotation=Quaternion.LookRotation(moveDirection);

          }

          else

          {

          Vector3 xzMove=movement;

          xzMove.y=0;

          if (xzMove.sqrMagnitude > 0.001f)

          {

          transform.rotation=Quaternion.LookRotation(xzMove);

          }

          }

          // We are in jump mode but just became grounded

          if (IsGrounded())

          {

          lastGroundedTime=Time.time;

          inAirVelocity=Vector3.zero;

          if (jumping)

          {

          jumping=false;

          SendMessage("DidLand", SendMessageOptions.DontRequireReceiver);

          }

          }

          }

          void OnControllerColliderHit(ControllerColliderHit hit)

          {

          // Debug.DrawRay(hit.point, hit.normal);

          if (hit.moveDirection.y > 0.01f)

          return;

          }

          float GetSpeed()

          {

          return moveSpeed;

          }

          public bool IsJumping()

          {

          return jumping;

          }

          bool IsGrounded()

          {

          return (collisionFlags & CollisionFlags.CollidedBelow) !=0;

          }

          Vector3 GetDirection()

          {

          return moveDirection;

          }

          public bool IsMovingBackwards()

          {

          return movingBack;

          }

          public float GetLockCameraTimer()

          {

          return lockCameraTimer;

          }

          bool IsMoving()

          {

          return Mathf.Abs(Input.GetAxisRaw("Vertical")) + Mathf.Abs(Input.GetAxisRaw("Horizontal")) > 0.5f;

          }

          bool HasJumpReachedApex()

          {

          return jumpingReachedApex;

          }

          bool IsGroundedWithTimeout()

          {

          return lastGroundedTime + groundedTimeout > Time.time;

          }

          void Reset()

          {

          gameObject.tag="Player";

          }

          }

          圖一張:

          三、根據需求,自己寫自己需要的控制方式。

          在本期訓練營中,主角超級瑪麗我才用了一種比較簡潔的控制方式,因為這種方式已經能夠滿足需求,該種方式就是前后左右移動的方式。該方式不需要添加charactercontroller,只需添加膠囊體就可。(PS:不過該方式有個缺點就是必須朝向固定,也就是只能朝向Z軸正方向)

          代碼如下:

          using UnityEngine;

          using System.Collections;

          public class MarioMove : MonoBehaviour

          {

          public float speed=5.0f;

          public static bool isGround;

          public static bool IsAllowJump;

          [SerializeField]

          float m_StationaryTurnSpeed=180;

          [SerializeField]

          float m_MovingTurnSpeed=360;

          float m_ForwardAmount;

          float m_TurnAmount;

          Vector3 m_GroundNormal;

          private Vector3 m_Move;

          private Transform m_Cam;

          private Vector3 m_CamForward;

          // Use this for initialization

          void Start()

          {

          // get the transform of the main camera

          if (Camera.main !=null)

          {

          m_Cam=Camera.main.transform;

          }

          else

          {

          Debug.LogWarning(

          "Warning: no main camera found. Third person character needs a Camera tagged \"MainCamera\", for camera-relative controls.");

          // we use self-relative controls in this case, which probably isn't what the user wants, but hey, we warned them!

          }

          }

          void OnCollisionEnter(Collision collision)

          {

          //if (collision.collider.tag=="Ground")

          if (collision.collider.tag !=null)

          {

          isGround=true;

          IsAllowJump=true;

          }

          else

          {

          isGround=false;

          }

          }

          // Update is called once per frame

          void Update()

          {

          float h=Input.GetAxis("Horizontal");

          float v=Input.GetAxis("Vertical");

          GetComponent<Rigidbody>().MovePosition(transform.position - new Vector3(h, 0, v) * speed * Time.deltaTime);

          if (isGround==true && Input.GetButton("Jump"))

          {

          if (IsAllowJump==true)

          {

          transform.GetComponentInChildren<Animation>().CrossFade("jump");

          GetComponent<Rigidbody>().MovePosition(transform.position - new Vector3(-h * 0.1f, -0.15f, -v * 0.1f));

          }

          }

          else if (Input.GetButtonUp("Jump"))

          {

          IsAllowJump=false;

          }

          else

          {

          if (Input.GetAxis("Vertical") > 0.5f ||

          Input.GetAxis("Vertical") < -0.5f ||

          Input.GetAxis("Horizontal") > 0.5f ||

          Input.GetAxis("Horizontal") < -0.5f)

          {

          transform.GetComponentInChildren<Animation>().CrossFade("run");

          }

          else if ((Input.GetAxis("Vertical") > 0.0f && Input.GetAxis("Vertical") < 0.5f) ||

          (Input.GetAxis("Vertical") > -0.5f && Input.GetAxis("Vertical") < 0.0f) ||

          (Input.GetAxis("Horizontal") > 0.0f && Input.GetAxis("Horizontal") < 0.5f) ||

          (Input.GetAxis("Horizontal") < 0.0f && Input.GetAxis("Horizontal") > -0.5f))

          {

          transform.GetComponentInChildren<Animation>().CrossFade("walk");

          }

          else

          {

          transform.GetComponentInChildren<Animation>().CrossFade("idle");

          }

          }

          if (m_Cam !=null)

          {

          // calculate camera relative direction to move:

          m_CamForward=Vector3.Scale(m_Cam.forward, new Vector3(1, 0, 1)).normalized;

          m_Move=v * m_CamForward + h * m_Cam.right;

          }

          else

          {

          // we use world-relative directions in the case of no main camera

          m_Move=v * Vector3.forward + h * Vector3.right;

          }

          Move(m_Move);

          }

          public void Move(Vector3 move)

          {

          // convert the world relative moveInput vector into a local-relative

          // turn amount and forward amount required to head in the desired

          // direction.

          if (move.magnitude > 1f) move.Normalize();

          move=transform.InverseTransformDirection(move);

          //CheckGroundStatus();

          move=Vector3.ProjectOnPlane(move, m_GroundNormal);

          m_TurnAmount=Mathf.Atan2(move.x, move.z);

          m_ForwardAmount=move.z;

          ApplyExtraTurnRotation();

          }

          void ApplyExtraTurnRotation()

          {

          // help the character turn faster (this is in addition to root rotation in the animation)

          float turnSpeed=Mathf.Lerp(m_StationaryTurnSpeed, m_MovingTurnSpeed, m_ForwardAmount);

          transform.Rotate(0, m_TurnAmount * turnSpeed * Time.deltaTime, 0);

          }

          }

          在跳躍的代碼部分,這樣寫的目的是實現了按跳躍鍵的時間長短跳的高度不同,和大家小時候玩的超級瑪麗游戲的感覺很像。

          附圖一張:

          原文鏈接:http://www.manew.com/thread-98040-1-1.html

          本文主要研究一下Java 9的Compact Strings

          Compressed Strings(Java 6)

          Java 6引入了Compressed Strings,對于one byte per character使用byte[],對于two bytes per character繼續使用char[];之前可以使用-XX:+UseCompressedStrings來開啟,不過在java7被廢棄了,然后在java8被移除

          Compact Strings(Java 9)

          Java 9引入了Compact Strings來取代Java 6的Compressed Strings,它的實現更過徹底,完全使用byte[]來替代char[],同時新引入了一個字段coder來標識是LATIN1還是UTF16

          String

          java.base/java/lang/String.java

          public final class String
           implements java.io.Serializable, Comparable<String>, CharSequence,
           Constable, ConstantDesc {
          ?
           /**
           * The value is used for character storage.
           *
           * @implNote This field is trusted by the VM, and is a subject to
           * constant folding if String instance is constant. Overwriting this
           * field after construction will cause problems.
           *
           * Additionally, it is marked with {@link Stable} to trust the contents
           * of the array. No other facility in JDK provides this functionality (yet).
           * {@link Stable} is safe here, because value is never null.
           */
           @Stable
           private final byte[] value;
          ?
           /**
           * The identifier of the encoding used to encode the bytes in
           * {@code value}. The supported values in this implementation are
           *
           * LATIN1
           * UTF16
           *
           * @implNote This field is trusted by the VM, and is a subject to
           * constant folding if String instance is constant. Overwriting this
           * field after construction will cause problems.
           */
           private final byte coder;
          ?
           /** Cache the hash code for the string */
           private int hash; // Default to 0
          ?
           /** use serialVersionUID from JDK 1.0.2 for interoperability */
           private static final long serialVersionUID=-6849794470754667710L;
          ?
           /**
           * If String compaction is disabled, the bytes in {@code value} are
           * always encoded in UTF16.
           *
           * For methods with several possible implementation paths, when String
           * compaction is disabled, only one code path is taken.
           *
           * The instance field value is generally opaque to optimizing JIT
           * compilers. Therefore, in performance-sensitive place, an explicit
           * check of the static boolean {@code COMPACT_STRINGS} is done first
           * before checking the {@code coder} field since the static boolean
           * {@code COMPACT_STRINGS} would be constant folded away by an
           * optimizing JIT compiler. The idioms for these cases are as follows.
           *
           * For code such as:
           *
           * if (coder==LATIN1) { ... }
           *
           * can be written more optimally as
           *
           * if (coder()==LATIN1) { ... }
           *
           * or:
           *
           * if (COMPACT_STRINGS && coder==LATIN1) { ... }
           *
           * An optimizing JIT compiler can fold the above conditional as:
           *
           * COMPACT_STRINGS==true=> if (coder==LATIN1) { ... }
           * COMPACT_STRINGS==false=> if (false) { ... }
           *
           * @implNote
           * The actual value for this field is injected by JVM. The static
           * initialization block is used to set the value here to communicate
           * that this static final field is not statically foldable, and to
           * avoid any possible circular dependency during vm initialization.
           */
           static final boolean COMPACT_STRINGS;
          ?
           static {
           COMPACT_STRINGS=true;
           }
          ?
           /**
           * Class String is special cased within the Serialization Stream Protocol.
           *
           * A String instance is written into an ObjectOutputStream according to
           * <a href="{@docRoot}/../specs/serialization/protocol.html#stream-elements">
           * Object Serialization Specification, Section 6.2, "Stream Elements"</a>
           */
           private static final ObjectStreamField[] serialPersistentFields=new ObjectStreamField[0];
          ?
           /**
           * Initializes a newly created {@code String} object so that it represents
           * an empty character sequence. Note that use of this constructor is
           * unnecessary since Strings are immutable.
           */
           public String() {
           this.value="".value;
           this.coder="".coder;
           }
          ?
           //......
          ?
           public char charAt(int index) {
           if (isLatin1()) {
           return StringLatin1.charAt(value, index);
           } else {
           return StringUTF16.charAt(value, index);
           }
           }
          ?
           public boolean equals(Object anObject) {
           if (this==anObject) {
           return true;
           }
           if (anObject instanceof String) {
           String aString=(String)anObject;
           if (coder()==aString.coder()) {
           return isLatin1() ? StringLatin1.equals(value, aString.value)
           : StringUTF16.equals(value, aString.value);
           }
           }
           return false;
           }
          ?
           public int compareTo(String anotherString) {
           byte v1[]=value;
           byte v2[]=anotherString.value;
           if (coder()==anotherString.coder()) {
           return isLatin1() ? StringLatin1.compareTo(v1, v2)
           : StringUTF16.compareTo(v1, v2);
           }
           return isLatin1() ? StringLatin1.compareToUTF16(v1, v2)
           : StringUTF16.compareToLatin1(v1, v2);
           }
          ?
           public int hashCode() {
           int h=hash;
           if (h==0 && value.length > 0) {
           hash=h=isLatin1() ? StringLatin1.hashCode(value)
           : StringUTF16.hashCode(value);
           }
           return h;
           }
          ?
           public int indexOf(int ch, int fromIndex) {
           return isLatin1() ? StringLatin1.indexOf(value, ch, fromIndex)
           : StringUTF16.indexOf(value, ch, fromIndex);
           }
          ?
           public String substring(int beginIndex) {
           if (beginIndex < 0) {
           throw new StringIndexOutOfBoundsException(beginIndex);
           }
           int subLen=length() - beginIndex;
           if (subLen < 0) {
           throw new StringIndexOutOfBoundsException(subLen);
           }
           if (beginIndex==0) {
           return this;
           }
           return isLatin1() ? StringLatin1.newString(value, beginIndex, subLen)
           : StringUTF16.newString(value, beginIndex, subLen);
           }
          ?
           //......
          ?
           byte coder() {
           return COMPACT_STRINGS ? coder : UTF16;
           }
          ?
           byte[] value() {
           return value;
           }
          ?
           private boolean isLatin1() {
           return COMPACT_STRINGS && coder==LATIN1;
           }
          ?
           @Native static final byte LATIN1=0;
           @Native static final byte UTF16=1;
          ?
           //......
          }
          
          • COMPACT_STRINGS默認為true,即該特性默認是開啟的
          • coder方法判斷COMPACT_STRINGS為true的話,則返回coder值,否則返回UTF16;isLatin1方法判斷COMPACT_STRINGS為true且coder為LATIN1則返回true
          • 諸如charAt、equals、hashCode、indexOf、substring等等一系列方法都依賴isLatin1方法來區分對待是StringLatin1還是StringUTF16

          StringConcatFactory

          實例

          public class Java9StringDemo {
          ?
           public static void main(String[] args){
           String stringLiteral="tom";
           String stringObject=stringLiteral + "cat";
           }
          }
          
          • 這段代碼stringObject由變量stringLiteral及cat拼接而來

          javap

          javac src/main/java/com/example/javac/Java9StringDemo.java
          javap -v src/main/java/com/example/javac/Java9StringDemo.class
          ?
           Last modified 2019年4月7日; size 770 bytes
           MD5 checksum fecfca9c829402c358c4d5cb948004ff
           Compiled from "Java9StringDemo.java"
          public class com.example.javac.Java9StringDemo
           minor version: 0
           major version: 56
           flags: (0x0021) ACC_PUBLIC, ACC_SUPER
           this_class: #4 // com/example/javac/Java9StringDemo
           super_class: #5 // java/lang/Object
           interfaces: 0, fields: 0, methods: 2, attributes: 3
          Constant pool:
           #1=Methodref #5.#14 // java/lang/Object."<init>":()V
           #2=String #15 // tom
           #3=InvokeDynamic #0:#19 // #0:makeConcatWithConstants:(Ljava/lang/String;)Ljava/lang/String;
           #4=Class #20 // com/example/javac/Java9StringDemo
           #5=Class #21 // java/lang/Object
           #6=Utf8 <init>
           #7=Utf8 ()V
           #8=Utf8 Code
           #9=Utf8 LineNumberTable
           #10=Utf8 main
           #11=Utf8 ([Ljava/lang/String;)V
           #12=Utf8 SourceFile
           #13=Utf8 Java9StringDemo.java
           #14=NameAndType #6:#7 // "<init>":()V
           #15=Utf8 tom
           #16=Utf8 BootstrapMethods
           #17=MethodHandle 6:#22 // REF_invokeStatic java/lang/invoke/StringConcatFactory.makeConcatWithConstants:(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;Ljava/lang/String;[Ljava/lang/Object;)Ljava/lang/invoke/CallSite;
           #18=String #23 // \u0001cat
           #19=NameAndType #24:#25 // makeConcatWithConstants:(Ljava/lang/String;)Ljava/lang/String;
           #20=Utf8 com/example/javac/Java9StringDemo
           #21=Utf8 java/lang/Object
           #22=Methodref #26.#27 // java/lang/invoke/StringConcatFactory.makeConcatWithConstants:(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;Ljava/lang/String;[Ljava/lang/Object;)Ljava/lang/invoke/CallSite;
           #23=Utf8 \u0001cat
           #24=Utf8 makeConcatWithConstants
           #25=Utf8 (Ljava/lang/String;)Ljava/lang/String;
           #26=Class #28 // java/lang/invoke/StringConcatFactory
           #27=NameAndType #24:#32 // makeConcatWithConstants:(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;Ljava/lang/String;[Ljava/lang/Object;)Ljava/lang/invoke/CallSite;
           #28=Utf8 java/lang/invoke/StringConcatFactory
           #29=Class #34 // java/lang/invoke/MethodHandles$Lookup
           #30=Utf8 Lookup
           #31=Utf8 InnerClasses
           #32=Utf8 (Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;Ljava/lang/String;[Ljava/lang/Object;)Ljava/lang/invoke/CallSite;
           #33=Class #35 // java/lang/invoke/MethodHandles
           #34=Utf8 java/lang/invoke/MethodHandles$Lookup
           #35=Utf8 java/lang/invoke/MethodHandles
          {
           public com.example.javac.Java9StringDemo();
           descriptor: ()V
           flags: (0x0001) ACC_PUBLIC
           Code:
           stack=1, locals=1, args_size=1
           0: aload_0
           1: invokespecial #1 // Method java/lang/Object."<init>":()V
           4: return
           LineNumberTable:
           line 8: 0
          ?
           public static void main(java.lang.String[]);
           descriptor: ([Ljava/lang/String;)V
           flags: (0x0009) ACC_PUBLIC, ACC_STATIC
           Code:
           stack=1, locals=3, args_size=1
           0: ldc #2 // String tom
           2: astore_1
           3: aload_1
           4: invokedynamic #3, 0 // InvokeDynamic #0:makeConcatWithConstants:(Ljava/lang/String;)Ljava/lang/String;
           9: astore_2
           10: return
           LineNumberTable:
           line 11: 0
           line 12: 3
           line 13: 10
          }
          SourceFile: "Java9StringDemo.java"
          InnerClasses:
           public static final #30=#29 of #33; // Lookup=class java/lang/invoke/MethodHandles$Lookup of class java/lang/invoke/MethodHandles
          BootstrapMethods:
           0: #17 REF_invokeStatic java/lang/invoke/StringConcatFactory.makeConcatWithConstants:(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;Ljava/lang/String;[Ljava/lang/Object;)Ljava/lang/invoke/CallSite;
           Method arguments:
           #18 \u0001cat
          
          • javap之后可以看到通過Java 9利用InvokeDynamic調用了StringConcatFactory.makeConcatWithConstants方法進行字符串拼接優化;而Java 8則是通過轉換為StringBuilder來進行優化

          StringConcatFactory.makeConcatWithConstants

          java.base/java/lang/invoke/StringConcatFactory.java

          public final class StringConcatFactory {
           //......
          ?
           /**
           * Concatenation strategy to use. See {@link Strategy} for possible options.
           * This option is controllable with -Djava.lang.invoke.stringConcat JDK option.
           */
           private static Strategy STRATEGY;
          ?
           /**
           * Default strategy to use for concatenation.
           */
           private static final Strategy DEFAULT_STRATEGY=Strategy.MH_INLINE_SIZED_EXACT;
          ?
           private enum Strategy {
           /**
           * Bytecode generator, calling into {@link java.lang.StringBuilder}.
           */
           BC_SB,
          ?
           /**
           * Bytecode generator, calling into {@link java.lang.StringBuilder};
           * but trying to estimate the required storage.
           */
           BC_SB_SIZED,
          ?
           /**
           * Bytecode generator, calling into {@link java.lang.StringBuilder};
           * but computing the required storage exactly.
           */
           BC_SB_SIZED_EXACT,
          ?
           /**
           * MethodHandle-based generator, that in the end calls into {@link java.lang.StringBuilder}.
           * This strategy also tries to estimate the required storage.
           */
           MH_SB_SIZED,
          ?
           /**
           * MethodHandle-based generator, that in the end calls into {@link java.lang.StringBuilder}.
           * This strategy also estimate the required storage exactly.
           */
           MH_SB_SIZED_EXACT,
          ?
           /**
           * MethodHandle-based generator, that constructs its own byte[] array from
           * the arguments. It computes the required storage exactly.
           */
           MH_INLINE_SIZED_EXACT
           }
          ?
           static {
           // In case we need to double-back onto the StringConcatFactory during this
           // static initialization, make sure we have the reasonable defaults to complete
           // the static initialization properly. After that, actual users would use
           // the proper values we have read from the properties.
           STRATEGY=DEFAULT_STRATEGY;
           // CACHE_ENABLE=false; // implied
           // CACHE=null; // implied
           // DEBUG=false; // implied
           // DUMPER=null; // implied
          ?
           Properties props=GetPropertyAction.privilegedGetProperties();
           final String strategy=props.getProperty("java.lang.invoke.stringConcat");
           CACHE_ENABLE=Boolean.parseBoolean(
           props.getProperty("java.lang.invoke.stringConcat.cache"));
           DEBUG=Boolean.parseBoolean(
           props.getProperty("java.lang.invoke.stringConcat.debug"));
           final String dumpPath=props.getProperty("java.lang.invoke.stringConcat.dumpClasses");
          ?
           STRATEGY=(strategy==null) ? DEFAULT_STRATEGY : Strategy.valueOf(strategy);
           CACHE=CACHE_ENABLE ? new ConcurrentHashMap<>() : null;
           DUMPER=(dumpPath==null) ? null : ProxyClassesDumper.getInstance(dumpPath);
           }
          ?
           public static CallSite makeConcatWithConstants(MethodHandles.Lookup lookup,
           String name,
           MethodType concatType,
           String recipe,
           Object... constants) throws StringConcatException {
           if (DEBUG) {
           System.out.println("StringConcatFactory " + STRATEGY + " is here for " + concatType + ", {" + recipe + "}, " + Arrays.toString(constants));
           }
          ?
           return doStringConcat(lookup, name, concatType, false, recipe, constants);
           }
          ?
           private static CallSite doStringConcat(MethodHandles.Lookup lookup,
           String name,
           MethodType concatType,
           boolean generateRecipe,
           String recipe,
           Object... constants) throws StringConcatException {
           Objects.requireNonNull(lookup, "Lookup is null");
           Objects.requireNonNull(name, "Name is null");
           Objects.requireNonNull(concatType, "Concat type is null");
           Objects.requireNonNull(constants, "Constants are null");
          ?
           for (Object o : constants) {
           Objects.requireNonNull(o, "Cannot accept null constants");
           }
          ?
           if ((lookup.lookupModes() & MethodHandles.Lookup.PRIVATE)==0) {
           throw new StringConcatException("Invalid caller: " +
           lookup.lookupClass().getName());
           }
          ?
           int cCount=0;
           int oCount=0;
           if (generateRecipe) {
           // Mock the recipe to reuse the concat generator code
           char[] value=new char[concatType.parameterCount()];
           Arrays.fill(value, TAG_ARG);
           recipe=new String(value);
           oCount=concatType.parameterCount();
           } else {
           Objects.requireNonNull(recipe, "Recipe is null");
          ?
           for (int i=0; i < recipe.length(); i++) {
           char c=recipe.charAt(i);
           if (c==TAG_CONST) cCount++;
           if (c==TAG_ARG) oCount++;
           }
           }
          ?
           if (oCount !=concatType.parameterCount()) {
           throw new StringConcatException(
           "Mismatched number of concat arguments: recipe wants " +
           oCount +
           " arguments, but signature provides " +
           concatType.parameterCount());
           }
          ?
           if (cCount !=constants.length) {
           throw new StringConcatException(
           "Mismatched number of concat constants: recipe wants " +
           cCount +
           " constants, but only " +
           constants.length +
           " are passed");
           }
          ?
           if (!concatType.returnType().isAssignableFrom(String.class)) {
           throw new StringConcatException(
           "The return type should be compatible with String, but it is " +
           concatType.returnType());
           }
          ?
           if (concatType.parameterSlotCount() > MAX_INDY_CONCAT_ARG_SLOTS) {
           throw new StringConcatException("Too many concat argument slots: " +
           concatType.parameterSlotCount() +
           ", can only accept " +
           MAX_INDY_CONCAT_ARG_SLOTS);
           }
          ?
           String className=getClassName(lookup.lookupClass());
           MethodType mt=adaptType(concatType);
           Recipe rec=new Recipe(recipe, constants);
          ?
           MethodHandle mh;
           if (CACHE_ENABLE) {
           Key key=new Key(className, mt, rec);
           mh=CACHE.get(key);
           if (mh==null) {
           mh=generate(lookup, className, mt, rec);
           CACHE.put(key, mh);
           }
           } else {
           mh=generate(lookup, className, mt, rec);
           }
           return new ConstantCallSite(mh.asType(concatType));
           }
          ?
           private static MethodHandle generate(Lookup lookup, String className, MethodType mt, Recipe recipe) throws StringConcatException {
           try {
           switch (STRATEGY) {
           case BC_SB:
           return BytecodeStringBuilderStrategy.generate(lookup, className, mt, recipe, Mode.DEFAULT);
           case BC_SB_SIZED:
           return BytecodeStringBuilderStrategy.generate(lookup, className, mt, recipe, Mode.SIZED);
           case BC_SB_SIZED_EXACT:
           return BytecodeStringBuilderStrategy.generate(lookup, className, mt, recipe, Mode.SIZED_EXACT);
           case MH_SB_SIZED:
           return MethodHandleStringBuilderStrategy.generate(mt, recipe, Mode.SIZED);
           case MH_SB_SIZED_EXACT:
           return MethodHandleStringBuilderStrategy.generate(mt, recipe, Mode.SIZED_EXACT);
           case MH_INLINE_SIZED_EXACT:
           return MethodHandleInlineCopyStrategy.generate(mt, recipe);
           default:
           throw new StringConcatException("Concatenation strategy " + STRATEGY + " is not implemented");
           }
           } catch (Error | StringConcatException e) {
           // Pass through any error or existing StringConcatException
           throw e;
           } catch (Throwable t) {
           throw new StringConcatException("Generator failed", t);
           }
           }
          ?
           //......
          }
          
          • makeConcatWithConstants方法內部調用了doStringConcat,而doStringConcat方法則調用了generate方法來生成MethodHandle;generate根據不同的STRATEGY來生成MethodHandle,這些STRATEGY有BC_SB、BC_SB_SIZED、BC_SB_SIZED_EXACT、MH_SB_SIZED、MH_SB_SIZED_EXACT、MH_INLINE_SIZED_EXACT,默認是MH_INLINE_SIZED_EXACT(可以通過-Djava.lang.invoke.stringConcat來改變默認的策略)

          小結

          • Java 9引入了Compact Strings來取代Java 6的Compressed Strings,它的實現更過徹底,完全使用byte[]來替代char[],同時新引入了一個字段coder來標識是LATIN1還是UTF16
          • isLatin1方法判斷COMPACT_STRINGS為true且coder為LATIN1則返回true;諸如charAt、equals、hashCode、indexOf、substring等等一系列方法都依賴isLatin1方法來區分對待是StringLatin1還是StringUTF16
          • Java 9利用InvokeDynamic調用了StringConcatFactory.makeConcatWithConstants方法進行字符串拼接優化,相比于Java 8通過轉換為StringBuilder來進行優化,Java 9提供了多種STRATEGY可供選擇,這些STRATEGY有BC_SB(等價于Java 8的優化方式)、BC_SB_SIZED、BC_SB_SIZED_EXACT、MH_SB_SIZED、MH_SB_SIZED_EXACT、MH_INLINE_SIZED_EXACT,默認是MH_INLINE_SIZED_EXACT(可以通過-Djava.lang.invoke.stringConcat來改變默認的策略)

          doc

          • String Compaction
          • JEP 254: Compact Strings
          • Java 9: Compact Strings
          • Compact Strings In Java 9
          • Java 9 Compact Strings Example
          • Evolution of Strings in Java to Compact Strings and Indify String Concatenation

          主站蜘蛛池模板: 国产一区二区三区不卡在线看| 久久久久久人妻一区精品| 久久国产视频一区| 亚洲一区二区影院| 色一情一乱一区二区三区啪啪高| 国产乱码精品一区二区三区| 国产激情一区二区三区在线观看 | 久久99精品一区二区三区| 色窝窝无码一区二区三区| 暖暖免费高清日本一区二区三区| 国产精品乱码一区二区三区| 91一区二区三区四区五区| 日美欧韩一区二去三区| 亚洲国产高清在线精品一区| 亚洲日本中文字幕一区二区三区 | 亚洲AⅤ无码一区二区三区在线 | 国产婷婷一区二区三区| 亚洲欧美成人一区二区三区 | 久久国产一区二区三区| 一区二区免费在线观看| 国模无码视频一区二区三区| 亚洲丰满熟女一区二区v| 99久久精品费精品国产一区二区 | 麻豆一区二区99久久久久| 国产在线不卡一区二区三区| 日韩精品一区二区三区视频| 中文精品一区二区三区四区 | 免费播放一区二区三区| 精品永久久福利一区二区| 人妻AV一区二区三区精品 | 免费看无码自慰一区二区| 国产99视频精品一区| 日韩精品一区二区三区中文版| 亚洲乱码av中文一区二区 | 亚洲av无码片区一区二区三区| 久久精品亚洲一区二区| 一区二区三区亚洲| 精品深夜AV无码一区二区| 大香伊人久久精品一区二区| 亚洲欧美日韩一区二区三区| 国产精品视频免费一区二区|