머신러닝스터디/2016/2016 07 09 - Revision history

Maintenance script: Repair batch-0005 pages from live compare

2026-03-27T00:44:57Z

Repair batch-0005 pages from live compare

← Older revision		Revision as of 00:44, 27 March 2026
Line 38:		Line 38:
	model.add(Dense(1, activation="sigmoid"))		model.add(Dense(1, activation="sigmoid"))

	model.compile(loss="binary_crossentropy", optimizer="adagrad", metrics=["accuracy"])		model.compile(loss="binary_crossentropy", optimizer="adagrad", metrics=["accuracy"])

	model.fit(X_train, y_train, batch_size=500, nb_epoch=100)		model.fit(X_train, y_train, batch_size=500, nb_epoch=100)
Line 44:		Line 44:
	pred = model.predict(X_test, batch_size=20000)		pred = model.predict(X_test, batch_size=20000)

	print (pred[0], y_test[0])		print (pred[0], y_test[0])
	print (pred[1], y_test[1])		print (pred[1], y_test[1])
	print (pred[2], y_test[2])		print (pred[2], y_test[2])

	=== Padding ===		=== Padding ===
Line 57:		Line 57:

	결과 예시		결과 예시
	array([ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,		array([ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,		0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,		0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
Line 133:		Line 133:
	9, 156, 252, 19, 235, 20, 28, 5, 104, 76, 7, 17, 169,		9, 156, 252, 19, 235, 20, 28, 5, 104, 76, 7, 17, 169,
	35, 2, 17, 23, 2, 7, 36, 2, 934, 56, 2, 6, 17,		35, 2, 17, 23, 2, 7, 36, 2, 934, 56, 2, 6, 17,
	891, 214, 11, 5, 2, 6, 92, 6, 33, 256, 82, 7], dtype=int32)		891, 214, 11, 5, 2, 6, 92, 6, 33, 256, 82, 7], dtype=int32)

	nb_words로 배열의 최대 길이를 지정하지 않으면 Embedding 단계에서 out of index 에러가 난다.		nb_words로 배열의 최대 길이를 지정하지 않으면 Embedding 단계에서 out of index 에러가 난다.
Line 141:		Line 141:
	Using Theano backend.		Using Theano backend.
	Epoch 1/10		Epoch 1/10
	22500/22500 [==============================] - 115s - loss: 0.6932 - acc: 0.5014		22500/22500 [==============================] - 115s - loss: 0.6932 - acc: 0.5014
	Epoch 2/10		Epoch 2/10
	22500/22500 [==============================] - 115s - loss: 0.6932 - acc: 0.5010		22500/22500 [==============================] - 115s - loss: 0.6932 - acc: 0.5010
	Epoch 3/10		Epoch 3/10
	22500/22500 [==============================] - 114s - loss: 0.6932 - acc: 0.5014		22500/22500 [==============================] - 114s - loss: 0.6932 - acc: 0.5014
	Epoch 4/10		Epoch 4/10
	22500/22500 [==============================] - 115s - loss: 0.6931 - acc: 0.5014		22500/22500 [==============================] - 115s - loss: 0.6931 - acc: 0.5014
	Epoch 5/10		Epoch 5/10
	22500/22500 [==============================] - 115s - loss: 0.6931 - acc: 0.5014		22500/22500 [==============================] - 115s - loss: 0.6931 - acc: 0.5014
	Epoch 6/10		Epoch 6/10
	22500/22500 [==============================] - 115s - loss: 0.6932 - acc: 0.5014		22500/22500 [==============================] - 115s - loss: 0.6932 - acc: 0.5014
	Epoch 7/10		Epoch 7/10
	22500/22500 [==============================] - 114s - loss: 0.6931 - acc: 0.5014		22500/22500 [==============================] - 114s - loss: 0.6931 - acc: 0.5014
	Epoch 8/10		Epoch 8/10
	22500/22500 [==============================] - 114s - loss: 0.6932 - acc: 0.5016		22500/22500 [==============================] - 114s - loss: 0.6932 - acc: 0.5016
	Epoch 9/10		Epoch 9/10
	22500/22500 [==============================] - 115s - loss: 0.6932 - acc: 0.5014		22500/22500 [==============================] - 115s - loss: 0.6932 - acc: 0.5014
	Epoch 10/10		Epoch 10/10
	22500/22500 [==============================] - 115s - loss: 0.6932 - acc: 0.5014		22500/22500 [==============================] - 115s - loss: 0.6932 - acc: 0.5014

	=== 학습 성공 ===		=== 학습 성공 ===
	Epoch 1/10		Epoch 1/10
	22500/22500 [==============================] - 14282s - loss: 0.6927 - acc: 0.5164		22500/22500 [==============================] - 14282s - loss: 0.6927 - acc: 0.5164
	Epoch 2/10		Epoch 2/10
	22500/22500 [==============================] - 10235s - loss: 0.6864 - acc: 0.5618		22500/22500 [==============================] - 10235s - loss: 0.6864 - acc: 0.5618
	Epoch 3/10		Epoch 3/10
	22500/22500 [==============================] - 3236s - loss: 0.6541 - acc: 0.6508		22500/22500 [==============================] - 3236s - loss: 0.6541 - acc: 0.6508
	Epoch 4/10		Epoch 4/10
	22500/22500 [==============================] - 3230s - loss: 0.5829 - acc: 0.7528		22500/22500 [==============================] - 3230s - loss: 0.5829 - acc: 0.7528
	Epoch 5/10		Epoch 5/10
	22500/22500 [==============================] - 3222s - loss: 0.5490 - acc: 0.7745		22500/22500 [==============================] - 3222s - loss: 0.5490 - acc: 0.7745
	Epoch 6/10		Epoch 6/10
	22500/22500 [==============================] - 3229s - loss: 0.5250 - acc: 0.7946		22500/22500 [==============================] - 3229s - loss: 0.5250 - acc: 0.7946
	Epoch 7/10		Epoch 7/10
	22500/22500 [==============================] - 3230s - loss: 0.5052 - acc: 0.8030		22500/22500 [==============================] - 3230s - loss: 0.5052 - acc: 0.8030
	Epoch 8/10		Epoch 8/10
	22300/22500 [============================>.] - ETA: 28s - loss: 0.4963 - acc: 0.8046		22300/22500 [============================>.] - ETA: 28s - loss: 0.4963 - acc: 0.8046
	(사실 다음날 보니 프로세스가 죽어있어서 Epoch 8/10 이후의 결과는 없음... 학습 실패임)		(사실 다음날 보니 프로세스가 죽어있어서 Epoch 8/10 이후의 결과는 없음... 학습 실패임)
	== 다음 시간에는 ==		== 다음 시간에는 ==
	* Coursera 동영상 week 7 보기		* Coursera 동영상 week 7 보기
	== 더 보기 ==		== 더 보기 ==

imported>rabierre at 17:33, 24 July 2016

2016-07-24T17:33:44Z

← Older revision		Revision as of 17:33, 24 July 2016
Line 1:		Line 1:
	~~[[pagelist(^(~~머신러닝스터디/2016~~))]]~~		[[머신러닝스터디/2016]]
			[[머신러닝스터디/2016/목차]]
	== 내용 ==		== 내용 ==
	* Embedding에는 word index가 필요함.		* Embedding에는 word index가 필요함.

175.223.30.236 at 12:42, 13 July 2016

2016-07-13T12:42:40Z

← Older revision		Revision as of 12:42, 13 July 2016
Line 177:		Line 177:
	Epoch 8/10		Epoch 8/10
	22300/22500 [============================>.] - ETA: 28s - loss: 0.4963 - acc: 0.8046		22300/22500 [============================>.] - ETA: 28s - loss: 0.4963 - acc: 0.8046
	(사실 다음날 보니 프로세스가 죽어있어서 Epoch 8/10 이후의 결과는 없음...)		(사실 다음날 보니 프로세스가 죽어있어서 Epoch 8/10 이후의 결과는 없음... 학습 실패임)
	== 다음 시간에는 ==		== 다음 시간에는 ==
	* Coursera 동영상 week 7 보기		* Coursera 동영상 week 7 보기
	== 더 보기 ==		== 더 보기 ==

114.108.98.54 at 05:23, 13 July 2016

2016-07-13T05:23:46Z

@@ Line 159: / Line 159: @@
   Epoch 10/10
 /22500 [==============================] - 115s - loss: 0.6932 - acc: 0.5014
 == 다음 시간에는 ==
 * Coursera 동영상 week 7 보기
 == 더 보기 ==

114.108.95.133 at 23:50, 11 July 2016

2016-07-11T23:50:06Z

← Older revision		Revision as of 23:50, 11 July 2016
Line 134:		Line 134:
	891, 214, 11, 5, 2, 6, 92, 6, 33, 256, 82, 7], dtype=int32)		891, 214, 11, 5, 2, 6, 92, 6, 33, 256, 82, 7], dtype=int32)

			nb_words로 배열의 최대 길이를 지정하지 않으면 Embedding 단계에서 out of index 에러가 난다.
	IndexError: index 4414 is out of bounds for size 1000		IndexError: index 4414 is out of bounds for size 1000

114.108.95.133 at 23:49, 11 July 2016

2016-07-11T23:49:21Z

@@ Line 46: / Line 46: @@
   print (pred[1], y_test[1])
   print (pred[2], y_test[2])
 == 다음 시간에는 ==
 * Coursera 동영상 week 7 보기
 == 더 보기 ==

114.108.95.133 at 03:15, 10 July 2016

2016-07-10T03:15:31Z

← Older revision		Revision as of 03:15, 10 July 2016
Line 10:		Line 10:
	** adamax 를 썼는데 accuracy가 50% 대에 머무름		** adamax 를 썼는데 accuracy가 50% 대에 머무름
	** tensorflow는 adamax를 제공하지 않음. keras 자체 구현됨([https://github.com/fchollet/keras/blob/c30432a665e23d598d2cec11094b0cc3cac2f46d/keras/optimizers.py#L356 code]).		** tensorflow는 adamax를 제공하지 않음. keras 자체 구현됨([https://github.com/fchollet/keras/blob/c30432a665e23d598d2cec11094b0cc3cac2f46d/keras/optimizers.py#L356 code]).

			* 적절한 batch size
			** batch size가 너무 작으면(e.g. 32) 학습이 오래 걸린다.
			** 반면 너무 크면 메모리를 많이 사용하게 된다.
	=== 코드 ===		=== 코드 ===
	import keras		import keras

114.108.95.133 at 03:05, 10 July 2016

2016-07-10T03:05:14Z

← Older revision		Revision as of 03:05, 10 July 2016
Line 9:		Line 9:
	* optimizer		* optimizer
	** adamax 를 썼는데 accuracy가 50% 대에 머무름		** adamax 를 썼는데 accuracy가 50% 대에 머무름
			** tensorflow는 adamax를 제공하지 않음. keras 자체 구현됨([https://github.com/fchollet/keras/blob/c30432a665e23d598d2cec11094b0cc3cac2f46d/keras/optimizers.py#L356 code]).
	=== 코드 ===		=== 코드 ===
	import keras		import keras

114.108.95.133 at 03:03, 10 July 2016

2016-07-10T03:03:43Z

← Older revision		Revision as of 03:03, 10 July 2016
Line 3:		Line 3:
	* Embedding에는 word index가 필요함.		* Embedding에는 word index가 필요함.
	** 초기에 Tokenizer로 word frequency를 input으로 썼는데 학습이 잘 안됨.		** 초기에 Tokenizer로 word frequency를 input으로 썼는데 학습이 잘 안됨.
	** ~~Embedding은 해당 단어의 유무를 one-hot coding하여 일차원 배열로 나타냄~~.		** [http://keras.io/layers/embeddings/]
	** Question: 단어의 빈도는 고려되지 않나?
	tokenizer = Tokenizer(nb_words=1000)		tokenizer = Tokenizer(nb_words=1000)
	X_train = tokenizer.sequences_to_matrix(X_train, mode="freq")		X_train = tokenizer.sequences_to_matrix(X_train, mode="freq")

114.108.95.133 at 02:59, 10 July 2016

2016-07-10T02:59:51Z

← Older revision		Revision as of 02:59, 10 July 2016
Line 1:		Line 1:
	[[pagelist(^(머신러닝스터디/2016))]]		[[pagelist(^(머신러닝스터디/2016))]]
	== 내용 ==		== 내용 ==
			* Embedding에는 word index가 필요함.
			** 초기에 Tokenizer로 word frequency를 input으로 썼는데 학습이 잘 안됨.
			** Embedding은 해당 단어의 유무를 one-hot coding하여 일차원 배열로 나타냄.
			** Question: 단어의 빈도는 고려되지 않나?
			tokenizer = Tokenizer(nb_words=1000)
			X_train = tokenizer.sequences_to_matrix(X_train, mode="freq")

			* optimizer
			** adamax 를 썼는데 accuracy가 50% 대에 머무름
	=== 코드 ===		=== 코드 ===
	import keras		import keras